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ABSTRACT 

A study examined statewide implementation of the 
Arizona Department of Education's vocational tr chnological education 
(ADE/VTE) model and the feasibility of using Arizona's performance 
standards in evaluating processes/outcomes at model sites. Data were 
collected from a pilot study of 12 sites and survey of 128 Arizona 
local education authorities (LEAs) . Although most schools surveyed 
believed they were implementing model programs, site visits 
established that many schools were not in fact replicating the 
ADE/VTE model. Conceptual /def ini t i onal difficulties with the ADE/VTE 
model and performance standards and problems in local implementation 
of tlie model and standards were identified. The following actions 
were recommended: simplify the model, increase technical assistance 
and sunport to LEAs toward implementing the VTE model, redesign the 
performance standards system to enable collection and reporting of 
valid and reliable student performance data, enhance technical 
assistance efforts to help sites col 1 ect /report more val id/rel iable 
data and use data for program improvement, and work toward designing 
an evaluation system incorporating recent f edera 1 /s tat e policy 
reforms. (Fourteen t ab 1 es/ f igures are included. Appendixes 
constituting approximately two- thirds of this report contain the 
following: list of ADE/VTE model-funded sites, site visit reports and 
checklists, site visit comparisons on program characteristics, and 
results of the LEA performance standards procedures survey.) (MN) 
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EXECUTIVE SUMMARY 



In Spring 1993, Morrison Institute for Public Policy, School of Public Affairs, Arizona State 
University, was awarded a contract to conduct a research project on behalf of the Arizona 
Department of Education, Division of Vocational Technological Education (ADE/VTE). 
Objectives of the project involved examining VTE Model implementation statewide, 
studying local procedures in conducting Performance Standards evaluations, and 
researching issues related to using ADE-housed Performance Standards data to evaluate 
"Model" sites (i.e., sites receiving ADE targeted fundi; for the implementation of the VTE 
Model). 

In fulfillment of these goals, Morrison Institute researchers conducted a survey of all LEAs 
regarding VTE Model implementation, site visits to both Model-funded sites and matched 
nonfunded sites, and a survey of LEAs regarding procedures used to complete 
Performance Standards evaluations. State and local documents were also analyzed. 

This report summarizes the research in three parts. Part One discusses the VTE Model and 
issues related to its implementation. As discussed in Part One, while most schools with 
state-approved, formula-funded programs believe they are implementing Model programs, 
data suggest that this may not be the case. Researchers identified conceptual and 
definitional problems with the Model (in terms of ADE criteria for a fully implemented 
Model program) as well as local implementation problems. These types of 
problems — Model problems and implementation problems— are interrelated and affect the 
replicability of the VTE Model and hence its reliability. 

Part Two discusses Performance Standards and Measures and local procedures related to 
conducting Performance Standards evaluations. Researchers identified problems with the 
Performance Standards data as well as local procedural problems in implementing 
Performance Standards evaluations. Similar to the VTE Mode! analysis, problems in 
defining Performance Standards and local implementation problems were found to be 
interrelated, suggesting a need to redefine the Performance Standards database. 

Part Three discusses issues pertinent to: a) the VTE Model, b) Performance Standards, and 
c) issues related to using Performance Standards for evaluating VTE Model sites. 
Regarding the latter, purposes of evaluation are discussed in terms of both outcomes and 
processes. Performance Standards and VTE Model characteristics are compared to these 
types of evaluations and critiqued. 

In summary, the report makes recommendations about the VTE Model, Performance 
Standards, and evaluation design. Recommendations are as follows: 

■ Simplify the Model. 

V Eliminate Model program characteristics that fail to distinguish between types of 
programs. 

V Reconsider and adjust several of the Model program characteristics. Specifically, 
department personnel should develop indicators or benchmarks for each program 
characteristic. This would allow for determining the extent of implementation 
rather than the "all or nothing" judgments currently reflected by the checklist. 



V Consider the amount of instructional time available for VTE courses, given general 
graduation requirements, and whether requiring three years of broad (Level I ) to 
specific (Level III) VTE training is most appropriate and realistic within that 
framework. 

V Better articulate Level I with Levels II and III. 



■ Increase technical assistance and support to local educational agencies toward 
implementing the VTE Model, and specifically: 

V Provide additional training to school-based VTE personnel on the six Model strands 
and on the ADE Model program characteristics. 

V Expand funding and/or technical assistance to develop appropriate multi-purpose 
environments that simulate those of business and industry. 



■ Redesign the Performance Standards system to enable both the collection and 
reporting of valid and reliable student performance data, 

V Continue to pursue the use of ASAP data as a means to document the academic 
skills of VTE program students. 

V Explore alternative means to document the attainment of students' occupational 
competencies, so as to produce comparable data. 

V Define terms that are essential to accurate data reporting. 

V Identify existing state and local databases that provide comparable data pertinent 
to Performance Standards (e.g., the state dropout reporting system). 



■ Enhance technical assistance efforts to enable sites to collect and report more valid 
and reliable data and to use data for program improvement. 

V Encourage/provide more training on appropriate curricwlar and assessment 
adaptations for special "needs students. 

V Provide increased technical assistance to site personnel to ensure their use of new 
and existing definitions pertinent to the collection and accurate reporting of 
Performance Standards data. 

V Enhance the training of local personnel on the use of existing state and local 
databases that provide comparable data pertinent to Performance Standards (e.g., 
the state dropout reporting system). 

V Encourage local district staff to share evaluation results with VTE staff, and assist 
local staff to meaningfully use evaluation data for program planning and 
improvement. 

■ Work toward designing an evaluation system(s) that incorporates recent federal and 
state policy reforms. 

V Examine the alignment of the Performance Standards (outcomes) and VTE Model 
Program Characteristics (processes) with respect to evaluation design. 

V Implement a process evaluation in addition to the Performance Standards 
outcomes evaluation with respect to VTE Model Program implementation. 
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INTRODUCTION 



In Spring 1993, Morrison institute for Public Policy, School of Public Affairs, Arizona State 
University, was awarded a contract (94-VF-SL-02-6.1 ) to conduct a research project on 
behalf of the Arizona Department of Education, Division of Vocational Technological 
Education (ADE/VTE). The research/evaluation study, originally titled "An Evaluation of 
Vocational Technological Education Model Implementation," had four primary objectives. 
The objectives involved examining VTE Model implementation statewide, studying locai 
procedures in conducting Performance Standards evaluations, and statistically analyzing 
ADE-housed Performance Standards data with the intent to design a method for using 
these data to longitudinally evaluate "Model" sites (i.e., sites receiving ADE targeted funds 
for the implementation of the VTE Model). 

As the study progressed, it became clear that objectives involving ADE-housed 
Performance Standards data could not be accomplished. This was due to difficulties 
experienced by the Department of Education in creating a database comprised of locally 
reported Performance Standards data. Therefore, in consultation with ADE/VTE personnel, 
Morrison Institute's project was amended to conduct a pilot study of 12 sites regarding 
VTE Model implementation in lieu of a statistical analysis of the database and design of a 
longitudinal evaluation. The primary purpose of the pilot study was to examine issues 
related to the local implementation of the VTE Model. Secondarily, the pilot study was 
used as a vehicle to explore issues pertinent to the evaluation of "Model sites." 

In fulfillment of the 1993-94 amended contract and related project objectives, this report is 
comprised of three main parts. 

■ Part One discusses issues pertinent to the statewide implementation of the VTE 
Model. The results of a statewide survey and the pilot study of 12 sites are 
documented in this section. 

■ Part Two discusses the findings of a survey designed to explore local procedures used 
in conducting Performance Standards evaluations. Results of the survey are used to 
assess the reliability and validity of Performance Standards data. 

■ Part Three explores the idea of using Performance Standards data as an evaluation tool 
for "Model" sites. In this section, reliability and validity issues are recapped as they 
peridin to both the VTE Model and the Performance Standards. The overall problems 
and prospects of evaluating "Model" sites using Performance Standards data are 
discussed. 



PART ONE 



IMPLEMENTING THE VOCA TIONAL 
TECHNOLOGICAL EDUCATION (VTE) MODEL IN ARIZONA 



This section of the report describes Arizona's VTE Model and discusses two studies of its 
implementation in Arizona schools completed by Morrison Institute research analysts. The 
first implementation study was designed to identify the status of all schools toward 
implementing the VTE Model. The second study involved site visits to 12 schools and was 
intended to assess issues relevant to the local implementation of the VTE Model. 

THE VTE MODEL 

During the 1 980s and culminating with the release of Workforce 2000 (Johnston & 
Packer, 1987), America's business and education communities became increasingly aware 
of the need to revitalize the training of the nation's workforce. During the same era, 
educational reformers began to emphasize content and student performance standards as 
means to improve schools. In keeping with these parallel movements, Arizona educators 
and business representatives alike began to reexamine beliefs and practices that had long 
shaped education in the state. 

This Arizona reexamination of educational practice spawned the development of the 
state's Essential Skills curriculum documents and its companion assessment program— the 
Arizona Student Assessment Program, or ASAP. In addition, a new concept for vocational 
programming evolved. 

The VTE Model was defined and developed over a three-year period (1987-1990) by 
ADE/VTE staff in concert with representatives of business and industry, district/school 
personnel, and state university staff. Initially referred to as the "Arizona Vocational 
Technological Education Curriculum Model" 1 , the VTE Model represents the state's vision 
of an articulated sequence of instruction for students in grade seven through 
postsecondary training. 

The VTE Model has three dimensions as shown in Figure 1 : scope of occupational training, 
sequence of training, and curricular content. Overall, this framework for vocational 
technological education reflects a shift away from an instructional model emphasizing 
content in a single training area (e.g., carpentry) to instruction that integrates multiple 
skills in preparing youth for work. 



1 See Snyder, Johnson, DeMuth & Tasker (1992), Final Report, Model Site Evaluation FY 1991-92, Tempe, AZ: 
Arizona State University, Vocational Education (p. 6; Figure 1). 
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Figure 1. 
Arizona's VTE Model 
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The first dimension of the Model focuses on the breadth and depth of occupational 
training. The Model emphasizes a broader approach to careers in its initial stages and 
moves to mors specific occupational training in the advanced stages. This progression is 
defined by a coherent sequence of instruction comprised of four levels of instruction— the 
second dimension of the Model. The four levels of instruction in the VTE Model, described 
in greater detail in Table 1-1, are Technological Explorations and Foundations (Level I), 
Technological Core (Level II), Technological Preparation (Level III), and Advanced 
Technology and Retraining (Level IV). 

Finally, six curricular strands have been incorporated into the Model from its inception. 
These curricular strands address skill development in thinking, applied academics, career 
development, life management, technology, and business economics/leadership. Strands 
are intended to be continuous themes across occupational areas and throughout four levels 
of instruction that comprise the VTE Model. There is consensus in the literature that 
supports the inclusion of these six strands in vocational education (cf . NCVRE, 1 989 
Education Summit). By comparison, there is not consensus on how these skill areas should 
be defined or incorporated as part of the curriculum. 
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Table 1-1. 



Vocational Technological Education (VTE) Model 
Levels of Instruction 



Level 


Name and Description 


I 


Technologies! Explorations and Foundations is intended to develop core skills in each of the 




six curricular strands and to provide occupational exploration experiences. 




Targets students in grades 7-9 


11 


Technological Core is intended to enhance core skill development and to develop 




occupational awareness and skills in one or more "occupational clusters" as follows: 




—Applied Biological Systems. 




— Business Management Technology 




—Human Services 




— Industrial Technology 




—Information Technology 




— Innovative Cluster (i.e., other) 




Targets students in grades 9-1 1 


111 


Technological Preparation is intended to reinforce core skills and develop specific 




occupational competence in one program or more (e.g., horticulture; electronics) offered in 




the occupational areas of Agriculture, Business, Health Occupations, Marketing, 




Occupational Home Economics, and Trade and Industry. 




Targets students in grades 10-12 


IV 


Advanced Technology and Retraining is intended to offer advanced occupational skills 




training. 




Targets postsecondary students 



VTE Model Program Characteristics 



Since developing the VTE Model, ADE/VTE personnel also have creatsd the Arizona Model 
for Vocational Technological Education Program Characteristics Checklist (henceforth, the 
Program Characteristics Checklist). These characteristics focus on "essential" program 
elements at each level of the VTE Model that apply to students in grades seven through 
twelve (i.e., Levels I, II, and III). Program elements are defined in the areas of organization, 
curriculum, instructional delivery, facilities and equipment, and teacher qualifications. 

The checklist has 20 criteria that define a "fully implemented VTE Model program." 
Morrison Institute analysts identified seven criteria that are unique to a particular level of 
the VTE Model. These criteria are differentiated from 13 that are common to all three 
levels of the VTE Model. "Unique" criteria are shown in Table 1-2, while "shared" criteria 
are given in Table 1-3. (Item numbers in Tables 1-2 and 1-3 reflect the placement of each 
item on ADE's Program Characteristics Checklist.) Unique and shared program 
characteristics were used to assess the implementation of the VTE Model in two separate, 
but related, studies conducted as part of Morrison Institute's 1993-94 contract with ADE. 
Descriptions of these studies comprise the remainder of Part One of this report. 
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Table 1-3. 

Shared VTE Model Program Characteristics 



I. Organization 

5. The experience has the support of the local school/district administration. 

6. An active advisory council assists in designing and implementing this experience. 

II. Curriculum 

7. Curriculum is competency-based with each student's attainment of each competency 
monitored. 

9. Curriculum includes all six of the Model "Strands." 

10. Curriculum provides for student leadership development, which can include the use of vocational 
student organizations. 

ill. Instructional Delivery 

12. The teacher serves as a facilitator of instruction, managing a variety of learning experiences for 
students. 

13. Students work in groups or teams frequently. 

14. Instructional activities include industry-realistic situations and materials, and are evaluated 
according to industry-related standards. 

15. The instructional delivery includes occupational experiences which can include shadowing and 
cooperative education. 

IV. Facilities and Equipment 

16. The instructional facility provides a multi-purpose environment with appropriate technology and 
support systems. 

17. The environment simulates one that would be found in business and industry. 

18. The equipment and materials utilized in the experience are determined on the basis of the 
curriculum and industry standards. 

V. Teacher Qualifications 

20. The teacher(s) have recent inservice to update and upgrade their teaching techniques and content 
knowledge. 
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IMPLEMENTATION STUDY 1: SELF-REPORTED IMPLEMENTATION OF THE VTE MODEL 



As part of Morrison Institute's 1993-94 contract, the Arizona Department of Education's 
Division of Vocational Technological Education (ADE/VTE) requested information about the 
status of all Arizona local educational agencies (LEAs) towards implementing the VTE 
Curriculum Model. Prior to this contract, this information was unavailable primarily because 
of a four-year history, of mixed state and local funding and implementation efforts. 

Chronologically, state appropriations for implementing the VTE Model were first made for 
the 1990-91 school year. Starting that year and continuing through 1992-93, ADE 
targeted fur Jing for "Model sites" using an annual competitive grant process. LEAs with 
formula-funded, state-approved VTE programs were allowed to apply for targeted "Model 
site" funds in order to implement a comprehensive VTE Model program (i.e., all three levels 
of the model that pertain to students in grades 7-1 2) or a Level I, II, or III program 
singularly or in combination. 

The 1993-94 school year witnessed a departure from previous years in that proposals 
were accepted only from LEAs planning to implement a comprehensive program in at least 
one occupational area (i.e., Agriculture, Business, Health Occupations, Marketing, 
Occupational Home Economics, or Trade and Industry). LEAs could request to use these 
grant funds to establish and implement all three levels of the Model or to implement one or 
two levels needed to complete the sequence of instruction necessary for a comprehensive 
program. This shift in funding was in keeping with the intent of ADE/VTE to move all 
districts in the state toward implementing comprehensive VTE Model programs. 

During the same time period (between 1990 and 1994), some districts and schools 
reportedly implemented all or parts of the VTE Model without additional state "Model site" 
funding. The history of VTE Model Implementation was such that after four years of the 
Model being in effect, there was not an accurate inventory of VTE Model programs in the 
state. Implementation Study 1 was designed to help create a state-of-the-art picture of 
Arizona's progress in implementing the VTE Model. 

A first step in this study was to develop a complete list of LEAs and their formula-funded 
(i.e., state-approved) VTE programs. This information was requested from ADE/VTE in July 
of 1993. Based on ADE records for FY 1992-93, it was determined that there were 1,137 
state-approved VTE programs. These programs were operating in — 

■ 1 00 districts including 1 72 schoo ls implementing: 

84 Agriculture Education programs, 

254 Business Education programs, 

38 Health Occupations Education programs, 

136 Marketing Education programs, 

192 Occupational Home Economics Education programs, and 

433 Trade and Industry Education programs 2 . 



2 Figures exclude Diversified Cooperative Education programs; Also may exclude districts/schools implementing 
only Level I and/or Level II of the VTE curriculum model. 
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A second step in the study was to identify which of the 100 districts and 172 schools 
with state-approved, formula-funded VTE programs also had received "Model site" grant 
funding. Analyses of ADE records indicated that Model site funding was awarded to— 

■ 47 districts including 82 schools implementing: 

39 Level I programs 
36 Level II programs 

(3 Applied Biological Systems, 2 Business Management Technology, 5 Human 
Services, 4 Industrial Technology, 4 Information Technology, 1 Innovative 
Cluster, 17 unspecified/unknown) 

45 Level III programs 

(3 Agriculture Education programs, 7 Business Education programs, 3 Health 
Occupations Education programs, 6 Marketing Education programs, 6 
Occupational Home Economics Education programs, 10 Trade and Industry 
Education programs, and 10 unspecified/ unknown) 

6 Comprehensive programs (Levels Nil), for a total of 

■ 126 "Model" programs (see Appendix A). 



The VTE Model Components Questionnaire 

In August and September of 1993, Morrison Institute researchers in consultation with 
ADE/VTE program staff developed a questionnaire for distribution to the 1 72 schools with 
at least one state-approved vocational program. LEAs were asked to indicate if they 
believed they had a Level I, Level II, and/or Level III program that met ADE-defined criteria 
according to the Program Characteristics Checklist. Questionnaires were customized for 
each of the 172 schools. Customization consisted of listing the state-approved VTE 
program(s) for that school according to ADE records. 

Questionnaires were distributed at a meeting of local vocational directors on September 
15, 1993 with a cover letter by the State Director for Vocational Education. Materials 
were mailed to local directors who were not in attendance at the September meeting. 
Responses were requested by September 30. Multiple contacts were made with local 
vocational directors through December 1993 by telephone, mail, and electronic 
communication. Follow-up was conducted both to obtain responses and to request 
clarification of reported information. 



Questionnaire Results 



This section summarizes key findings from the VTE Model Components Questionnaire. 3 

Ninety-four of the 100 districts responded to the survey for a 94 percent district response 
rate. Among these districts, 160 of the 172 schools responded fcr a school response rate 
of 93 percent 4 . Of the 160 responding schools: 

■ 92 schools reported fully implementing a VTE Model Level I program. 

■ 113 schools reported fully implementing a total of 289 VTE Model Level II 
programs. 

■ 1 32 schools reported fully implementing a total of 679 VTE Model Level III 
programs. 

The 679 reported Level II! Model programs represent 60 percent of ail formula-funded, 
state-approved programs. This indicates that a majority of LEAs believe that they have one 
or more programs that meet ADE "Model program" criteria at Level III. By occupational 
area, districts reported implementing — 



■ 66 Level II! Model programs in Agriculture Education 
(79% of the state-approved total) 

■ 165 Level III Model programs in Business Education 
(65% of the state-approved total) 

■ 28 Level III Model programs in Health Occupations Education 
(74% of the state-approved total) 

■ 71 Level III Model programs in Marketing Education 
(52% of the state-approved total) 



■ 80 Level III Model programs in Occupational Home Economics Education 
(42% of the state-approved total) 

■ 269 Level III Model programs in Trade and Industry Education 
(62% of the state-approved total). 



Supporting documentation and additional information on this study can be found in the December 1993 report 
Implementing the Vocational Technological Education Model in Arizona by Vandegrift, J.A., Noble, A.J., and Sullivan, 
H.J. Tempe, AZ: Arizona State University, Morrison Institute for Public Policy. 

4 Six districts, including eight schools, did not respond at all to the survey: Benson UHSD (Benson Union High 
School); Chandler Unified District (Chandler High School, John M. Anderson Junior High School, Willis Junior High 
School); Gilbert Unified District (Gilbert High School); Grand Canyon Unified District (Grand Canyon High School); 
Patagonia UHSD (Patagonia Union High School); and Santa Cruz Vallev Unified District (Calabasas School). In 
addition, the following four schools did not respond: Chaparral High School (Scottsdale Unified District) and Cholla, 
Rincon, and University High Schools (Tucson Unified). 
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The distribution and configuration of all LEA-reported Model programs is shown in Table 1- 
4. 



Table 1-4. 

Number of Schools Reporting VTE Model Programs by Level (N = 160) 



Level I 



Level II 



Level III 



Level I 
Level II 
Level Ml 



Levels I, II and 111 in place — Comprehensive programs 

(72 schools) 




Level I only 
(8 schools) 


Levels I and II 
(4 schools) 


Levels I and III 
(8 schools) 


i 

■■■■ 


Level II only 
(1 school) 


Levels II and 111 
(36 schools) 


wMmmmmmm 


Level III only 
(16 schools) 








No "Model" Level I, II, or III programs in place (15 schools) 



This table reveals that 72 schools (45 percent of the schools responding) report 
implementing a Level l-li-lll "comprehensive" Model program in at least one occupational/ 
program area. 

The next three rows of Table 1-4 reveal that a total of 73 schools report implementing one 
or two, but not all three, levels of the VTE Model. That is, eight schools report having a 
Level I Model program only, four report having Levels I and II, and so on. 

Finally, 15 schools report that they do not have a program that meets ADE-defined criteria 
at any level of the VTE Curriculum Model. Typically, these 15 schools reported coursework 
in an occupational area (e.g., typing) but did not consider these courses as Model 
programs. 

In summary, this study was designed to determine the total number of LEAs that believe 
they are implementing one or more levels of the VTE Model. The population of LEAs 
studied consisted of 172 schools in 100 districts identified as having state formula-funded 
VTE programs. Based on self-reported data, the status of all 172 schools toward 
implementing the VTE model is summarized in Table 1-5. 
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Table 1-5. 

Status of All LEAs with VTE Programs Toward Implementing the VTE Model (N = 172) 



Implementation Status # schools 9 

At least one comprehensive occupational training program (i.e., a 72 
program with a Level Mil sequence of instruction) 

Either one or two levels of the VTE Model 73 
No VTE Model programs in place 15 
Unknown (No response) 1 2 



a Numbers are unduplicated, even though schools might offer multiple training programs in various stages of 
implementation. Descriptors were applied by default — If not at least one comprehensive program, then at 
least one partial program, etc. 



Table 1-5 shows that there are a total of 145 schools (72 + 73) that believe they are 
implementing one or more levels of the VTE Model according to ADE-defined criteria. 
These schools report high numbers of Level I, Level II, and Level 111 Model programs (92, 
289, and 679, respectively). Specifically at Level III, over half of all LEA respondents 
believe they offer Model programs in accordance with ADE standards. This finding holds 
true in all occupational areas except for Occupational Home Economics Education. 

Researchers also examined the extent to which "Model site" funding is linked with the 
reported implementation of comprehensive programs. As previously noted, ADE has 
funded 47 districts since 1 990 for the express purpose of implementing one or more levels 
of the VTE Model toward establishing comprehensive programs. 

On the questionnaire, a comprehensive program was reported in 52 districts containing 72 
schools. Comparing these 52 districts with a list of districts that have been funded as 
Model sites (Appendix A), only 29 of the 52 received this additional state funding. This 
means that 23 districts developed comprehensive programs without state "Model site" 
funding. It also means that 18 Model-funded districts reported not offering all three levels 
of the VTE Model, even though some were funded explicitly to do so. 

Overall, the results of Implementation Study 1 raised a number of questions regarding the 
implementation of the VTE Model in Arizona such as: 

■ What does a comprehensive program look like? 

■ What barriers exist that prevent some districts from implementing comprehensive 
orograms? 

■ To what extent do ADE "Model site" grants contribute to the full implementation of 
the VTE Model? 
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IMPLEMENTATION STUDY 2: SITE VISITS TO VTE PROGRAMS 



Implementation Study 2— Site Visits to VTE Programs— was conducted to examine the 
implementation of the VTE Model in a sample of 1 2 school districts. The study was 
designed as a follow-up to the survey of 172 schools described in Study 1 . 

Site Selection and Site Visit Methods 

Sites were chosen to represent a balance across four characteristics. 

■ Comprehensive versus Non-Comprehensive programs: Half of the schools selected for 
study had reported on the VTE Model Components Questionnaire that *hey offered a 
Level l-II-lll sequence of instruction meeting ADE criteria for a fully implemented Model 
program; half reported not offering a comprehensive program. 

■ ADE Model-funded versus Not Model-funded sites: Half of the sites chosen had 
received state grant funding toward implementing a comprehensive program: half had 
not. Of the six state-funded sites, three were self-described comprehensive programs 
and three were not. Of the six nonfunded sites, three were self-described 
comprehensive programs and three were not. 

In order to more equitably compare the three state-funded comprehensive programs 
with the three state-funded non-comprehensive programs, sites were matched with 
respect to the number of years funded (i.e., four years, three years, or two years). 

■ Urban/Metropolitan versus Rural/Reservation areas: Sites were selected to represent 
VTE programs statewide. Four sites represent urban/metropolitan areas and eight were 
chosen to illustrate rural/reservation sites. Eight of Arizona's 15 counties are 
represented by these sites including Cochise, Coconino, Gila, Maricopa, Mohave, 
Navajo, Pinal, and Yuma counties. (One Pima county school dropped out of this 
voluntary £ dy and was replaced by a Maricopa county school.) 

■ Occupational/Program areas: To the degree possible, sites were matched in terms of 
the kind of VTE program visited (i.e., with respect to both the occupational area and 
specific program area). Thus, researchers compared a comprehensive Agricultural 
Business and Management program with a non-comprehensive Agricultural Business 
and Management program and so forth. In all cases, sites were matched with respect 
to occupational areas. In some cases, specific program areas could not be matched. 
Based on the combination of all variables, five occupational areas were studied. No 
matched Marketing Education programs were identified. 

A list of the 12 schools and synopsis of site characteristics is shown in Table 1-6 on the 
next page. 
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Table 1-6. 



VTE Program Site Visits 







Comprehensive 


Non-Comprehensive 






Programs 


Programs 




Urban/ 


■ Peoria Unified 


■ Deer Valley Unified 




Metropolitan 


(4- vear<s fundina) 


(4- vear^ fundino) 






Site/Proaram * 


Site/Program : 






Ironwood HS/ 


Deer Valley VTech/ 






Health Hrn matinnQ — 
It Cat LI 1 \J \*\*KJ JJa LHJ 1 lo 


ncoi li i wi*i*u}jaLiui io 






Nursing Assistant 


Nursing Assistant 


ADE 


Rural/ 


■ Mohave UHSD 


» Flagstaff Unified 


"Model-funded" 


Reservation 


{3 years funding) 


{3 years funding) 






Site/Program: 


Site/Program: 






Kingman HS/ 


Coconino HS/ 






Trade and Industry- 


Trade and industry- 






Video Production 


Electronics 






■ Buckeye UHSD 


* Valley UHSD 






{2 years funding) 


(2 years funding) 






Site/Program: 


Site/Program: 






Buckeye Union HS/ 


Valley Union HS/ 






Agriculture— 


Agriculture — 






DUolMcoo Ol Ivldl Say CI 1 1 CI 1 L 


Rifcln^cc Rj anan pmpnt 
DUoi I icoo oi i vi ai lay ci 1 1 ci 1 1 




Urban/ 


■ Fountain Hills Unified 


■ Tuma UnoU 




Metropolitan 


Site/Program: 


Site/Prog ram ' 








Fountain Hills Jr/Sr HS/ 


Kofa HS/ 






i raae anu inuustry — 


i raue anu inuusiry — 


Not "Model-funded" 




Radio & TV Production 


Drafting 




Rural/ 


■ Miami Unified 


■ Apache Junction 




Reservation 




Unified 






Site/Program: 


Site/Program: 






Miami HS/ 


Apache Junction HS/ 






Home Economics- 


Home Economics- 






Child Development 


Food Production 






■ Whiteriver Unified 


■ Holbrook Unified 






Site/Program: 


Site/Program: 






Alchesay HS/ 


Holbrook HS/ 






Business- 


Business ~- 






Administrative Support 


Administrative Support 
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Information about the selected program at each site was collected by conducting a 
structured interview with one or more administrators and/or faculty members associated 
with the program, reviewing program documents, touring the program facilities and, in 
some cases, talking with students from the program. The structured interview was based 
in large part on the Arizona Model for Vocational Technological Education Program 
Characteristics Checklist. Information from the site visits and interviews wl* used to 
complete a site visit report and program characteristics checklist for each program. 

Level III programs were selected as the focus area for the visit to each site because they 
represent the highest level of a comprehensive program and, therefore, provide a good 
perspective for examining the overall vocational technological education experience for 
students. Because all 1 2 visits were to high schools, more complete information often was 
available for Level II and Level III than for Level I. 

Individual site visit reports and accompanying checklists are contained in Appendix B. Data 
for all 1 2 programs are presented in the Site Visit Results section that follows. 

Site Visit Results 

This section addresses several separate analyses of site visit data. Data are summarized 
for all 1 2 programs collectively, with distinctions drawn between comprehensive and non- 
comprehensive programs (using the districts' classification of their own programs as 
comprehensive or non-comprehensive). Differences are also discussed between ADE- 
funded and nonfunded programs. Finally, data from all sites are analyzed in terms of 
shared and unique program characteristics. 

Analyses by School and Program Type 

Table 1-7 on the following page reflects the judgments of Morrison Institute researchers on 
ADE-defined Program Characteristics by the type of program, school, and ADE/VTE Model 
level for the 12 schools visited. Each individual program was judged on 61 program 
characteristics — 20 for Level I, 21 for Level II (because one criterion has two parts), and 
20 for Level III. 

As explained a the bottom of the table, the three numerical entries for each school 
represent "yes-no-not sure" judgments with respect to the 61 program characteristics 
examined. "Yes" entries indicate that a program fully met the ADE criterion; "No" entries 
indicate a partial implementation or absence of a characteristic. "Not sure" entries are used 
in cases where the site evaluator did not have the opportunity to verify the presence or 
absence of a characteristic, or when data were ambiguous. The larger number of "no" and 
"not sure" judgments for Level I is due to the fact that most Level I courses are in 
elementary or middle/junior high school sites not visited by the researchers or that the site 
does not offer a Level I program. 
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Table 1-7. 

Program Characteristics by Program Type, School and Model Level 



Self-Classification 



School No. 



Level and # Characteristics 



(20) 



II 

(21) 



III 

(20) 



Totals 
(61 possible 
characteristics 
per site) 



Cornprehensive- 
Model-funded 



1 
2 
3 



yes-no-ns 

* 3-2-15 
10-2-8 
14-3-3 



yes-no-ns 

16-4-1 
18-1-2 
15-3-3 



yes-no-ns 

17- 3-0 

18- 1-1 

19- 1-0 



yes-no-ns 

36-9-16 
46-4-11 
48-7-6 



Comprehensive- 
Not Model-funded 



4 
5 
6 



4-14-2 
5-6-9 
12-2-6 



6-14-1 
1 6-3-2 
18-1-2 



9-10-1 
15-5-0 
17-2-1 



19-38-4 
36-14-11 
47-5-9 



Non-comprehensive- 
Model-funded 



7 
8 
9 



18-2-0 
2-18-0 
0-20-0 



20-1-0 
10-6-5 
14-5-2 



19-1-0 
18-2-0 
16-3-1 



57-4-0 
30-26-5 
30-28-3 



Non-comprehensive- 
Not Model-funded 



10 
11 
12 



5-9-6 
7-12-1 
0-20-0 



12-7-2 
10-10-1 
15-3-3 



15-5-0 
14-6-0 
18-1-1 



32- 21-8 
31-28-2 

33- 24-4 



Totals by Type 



Comprehensive 

Non- 
comprehensive 



48-29-43 



32-81-7 



89-26-11 
81-32-13 



95-22-3 



100-18-2 



232-77-57 



213-131-22 



Model-funded 
Nonfunded 



47-47-26 
33-63-24 



93-20-13 
77-38-1 1 



107-11-2 
88-29-3 



247-78-41 
198-130-38 



Overall Totals 



80-110-50 



170-58-24 



195-40-5 



445-208-79 



•Entries in the table represent judgements by Morrison Institute researchers on program characteristics for 
each school. For example, the 3-2-15 entry for School 1, Level I indicates that School 1 received 3 "yes" 
checks (i.e., had 3 of the 20 characteristics) on the Program Characteristics Checklist for Level I, two "no" 
checks and 1 5 "not sure" checks. The "Totals by Type" show the total number of "yes-no-not sure" 
responses across all six schools of each type — comprehensive and non-comprehensive, and Model-funded and 
not Model-funded. The "Overall Totals" show the total numbers for all 12 schools. 
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Data aggregated at the bottom of the table permit comparisons of comprehensive and non- 
comprehensive sites and funded and nonfunded sites. The complete data on an item-by- 
item basis for each individual school is contained with the report for that school in 
Appendix B. Summary tables that compare a) comprehensive and non-uomprehensive and 
b) funded and non-funded sites also are provided for reference in Appendix C. 

School Findings: Examination of the data for each school reveals that none of the 1 2 
schools had all characteristics for a fully implemented VTE Mode! program. School #7, 
which was self-described as a non-comprehensive program, came closest to having all 61 
of ADE's program characteristics with 57 "yes" ratings, four "no" ratings, and no "not 
sure" ratings. School #4, self-described as a comprehensive program, was farthest from 
having all of the ADE program characteristics, with only 19 of 61 characteristics receiving 
"yes" ratings. 

Notwithstanding the number of "not sures" at Level I, it is clear both from examination of 
the researcher judgments across levels in the table and from the overall totals at the 
bottom that programs have progressively more Model program characteristics as they 
move from Level I to Levels II and III. This finding holds true regardless of program type 
(i.e., funded or nonfunded, comprehensive or non-comprehensive). 

Comprehensive and Non-comprehensive Program Comparisons: In general, for the 1 2- 
school sample, schools that reported themselves as having a comprehensive program met 
slightly more of the ADE criteria for a fully implemented VTE Model program than schools 
that reported themselves as non-comprehensive (with 232 yes ratings compared to 213). 
Overall, three of the comprehensive program sites (Schools #2, #3, and #ti had at least 
two-thirds of the Model characteristics, and two sites (Schools #1 and #5) had over half. 
By comparison, five of the non-comprehensive sites (Schools #8-12) had approximately 
half of the program characteristics. 

The self-described comprehensive programs are more "model-like" overall and for Levels I 
and II, but not Level III. That is, the higher the level, the more the differences between 
comprehensive and non-comprehensive programs in terms of Model program 
characteristics diminish. In fact, at Level III, self-described non-comprehensive programs 
maintain a slight advantage over self-described comprehensive programs in terms of "yes" 
ratings (with 100 yes ratings compared to 95). 

Each of the self-reported non-comprehensive programs considered themselves as non- 
comprehensive primarily because of their Level I program (cf. Appendices C and D). 
Of the six sites, two had no Level I program at ail, two had district-defined Level I 
programs that school personnel believed did not meet ADE criteria for a Model program, 
and two had ADE-like Level I programs that staff believed did not meet ADE criteria. This 
accounts for the high number of "no" ratings for Level I programs at non-comprehensive 
sites. 

Specific differences between comprehensive and non-comprehensive programs for each 
program characteristic at each level of the Model are presented in detail in Appendix C and 
summarized here briefly. Comparing what does exist at Level I, comprehensive Level I 
programs tended to be year-long, broader exploration courses which utilized ADE draft 
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Level I curricula. Comprehensive Level I programs also reported monitoring student 
competency attainment to a greater degree than non-comprehensive programs. 

At Level II, differences are not as pronounced between comprehensive and non- 
comprehensive programs as they are in Level I. Nevertheless, comprehensive programs 
appear to be more cognizant of developing skills for a cluster of related occupations and to 
be offered for a full year. Similar to their Level I counterparts, Level II programs in 
comprehensive sites report monitoring student competency attainment more than in non- 
comprehensive Level II programs. Level II comprehensive programs also appear to have 
better facilities and equipment. No noteworthy distinctions between comprehensive and 
non-comprehensive Levei III programs were observed in the site visits. 

Notably, each group has an "outlier" site (a site where scores are beyond the "average" or 
typical range of scores for that group). School #4 in the comprehensive category had the 
lowest number of "yes" entries of any school, with 19 of 61 program characteristics rated 
as "yes." School #7 in the non-comprehensive category had the highest number of "yes" 
entries of any school, with 57 of 61 program characteristics rated as "yes." Site 
observations and interviews help explain the ratings of these two schools. In School #7, 
personnel rated their program as non-comprehensive because they are extremely well- 
versed in the ADE criteria for a fully implemented Model program and knew they did not 
meet all criteria. For example, their Level I experience is a year-long, district-requirement 
for all students except for those enrolled in band. This excludes the school from meeting 
the "100% served" criterion measure (Organization #2 on the Program Characteristics 
Checklist). Similarly, transfer students are enrolled in Level II and Level II! programs; 
therefore, staff exclude themselves from meeting the Organization #2 item for Levels II 
and III as well. 

Conversely, School #4 staff rated themselves as having a comprehensive program 
irrespective of the ADE Program Characteristics Checklist. School #4 staff were not 
familiar with the ADE/VTE Model, ADE-approved curriculum materials, ADE strands, and 
so forth. They said they had been "left out of the loop." Nonetheless, staff had created a 
competency-based program of their own which they considered to be "comprehensive." 

The two extremes represented by Schools #4 and #7 — one good program rating itself 
severely and another adequate program rating itself leniently— raise the issue of the 
subjectivity of ADE-defined Model Program Characteristics. This topic is addressed in the 
discussion section. 

Model-funded and Not Model-funded Program Comparisons: Based on totals by type in 
Table 1-7, model-funded programs meet more of the ADE criteria for a fully implemented 
Model program than do programs that have not received Model funding. Model-funded 
programs received more "yes" ratings than programs not funded at all levels of the VTE 
Model (247 yes ratings compared to 198). 

Specific differences between Model-funded and nonfunded programs for each program 
characteristic at each level of the Model are presented in detail in Appendix C and 
summarized here briefly. Level I programs at Model-funded sites tended to be year-long, 
broader exploration courses which utilized AD(- draft Level I curricula, incorporating the six 
Model strands. Funded Level I programs also reported monitoring student competency 
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attainment to a greater degree than programs that have not received Model funding. In 
funded programs, a greater number of teachers of the Level I experience are vocationally 
certified. 

At Level II, Model-funded programs were more cognizant of developing skills for a cluster 
of related occupations, had in place more than one Level II experience (thus allowing 
students to participate in more than one cluster, if desired), and focused instructional 
delivery on the occupational areas associated with a particular cluster. Model-funded Level 
II programs reported monitoring student competency attainment, incorporating the six 
Model strands, and including occupational experiences for students more than Level II 
programs that had not received Model funds. Level II Model-funded programs were 
observed to have better facilities and equipment. 

There are no noteworthy distinctions between Model-funded and nonfunded Level III 
programs in terms of program characteristics "unique" to Level 111. However, on common 
criteria (shared across levels), Level III funded programs incorporated the six Model strands 
and employed instructional delivery techniques aligned with industry-related standards, 
including occupational experiences. Furthermore, Model-funded Level III programs 
maintained an advantage over nonfunded programs with respect to facilities and 
equipment. 

Facilities and equipment appear to be one of the main features that distinguish between 
funded and nonfunded sites. Based on interview data, personnel at most funded sites 
indicated that ADE funds allowed them to purchase state-of-the-art equipment. 
Furthermore, observations of facilities suggest that funded sites have more and newer 
equipment than nonfunded sites. 

Analyses by Program Characteristics 

The next set of analyses use site visit data for all 12 sites. The discussion focuses on 
issues related to implementing the VTE Model as a whole (in terms of shared program 
characteristics) and issues that are pertinent to each level of the Model (in terms of unique 
program characteristics). 

Shared Program Characteristics: Table 1-8 on the following page shows the Morrison 
Institute researchers' judgments regarding the presence or absence of each of the shared 
program characteristics on the ADE Program Characteristics Checklist. Table 1-8 allows 
one to tentatively identify overall implementation problems and problems at each level of 
the Model that are associated with shared program characteristics. "Problem" areas are 
identified as those where less than half of the programs studied have "yes" entries and are 
indicated by the shaded areas on Table 1-8. The absence of a "yes" rating suggests that 
programs are not "in compliance" with the ADE criterion. 
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As derived from Table 1-8, nine of the thirteen shared characteristics at Level I appear as 
areas of concern. In rank order from most to least problematic, Level I programs tend not 
to: 

■ provide for student leadership development {Curriculum #10) or occupational 
experiences (Instructional Delivery #15) 

■ involve an active advisory council (Organization #6), include all six Model strands 
(Curriculum #9), or simulate a business and industry environment (Facilities and 
Equipment #1 7) 

■ include industry-realistic situations and materials, evaluated according to industry- 
related standards (Instructional Delivery #14) or use equipment and materials 
determined on the basis of the curriculum and industry standards (Facilities and 
Equipment #18) 

■ offer a competency-based curriculum and monitor students' competency attainment 
(Curriculum #7) or use a multi-purpose environment (Facilities and Equipment #16). 

At Level II, only three characteristics are not met by at least half of the programs. Level II 
problem areas relate to the integration of the six Model curriculum strands (Curriculum #9), 
the provision of occupational experiences (Instructional Delivery #15), and the 
incorporation of industry-realistic situations and materials (Instructional Delivery #14). 

Finally, at Level III, the 12 programs visited are generally not in compliance with only one 
shared characteristic— the integration of the six Model curriculum strands (Curriculum #9). 
Only two of the 12 programs knowingly incorporated all six of the Model stands. 

Following the totals provided at the right-hand side of Table 1-8, for all programs across all 
levels of program implementation, four characteristics appear problematic: 

■ the integration of the six curriculum strands (Curriculum #9) 

■ the inclusion of occupational experiences (instructional Delivery #15) 

■ the use of industry-realistic situations and materials evaluated according to industry- 
realistic standards (Instructional Delivery #14) 

■ the provision of a simulated business and industry environment (Facilities and 
Equipment #1 7), 



Overall, the 1 2 programs were more successful in satisfying shared characteristics for 
Level III (with 123 yes, 20 no, and 4 not sure entries) than for Level II (103 yes, 32 no, 
and 21 not sure), and more successful in satisfying characteristics for Level II than for 
Level I. 



Unique Program Characteristics: Table 1-9 on the following page shows the Morrison 
Institute researchers' judgments regarding the presence or absence of each of the unique 
program characteristics on the ADE Program Characteristics Checklist Table 1-9 allows 
one to identify implementation problems unique to each level of the Model with respect to 
the sample. As in the previous analysis, "problem" areas are identified as those where less 
than half of the programs studied have "yes" entries, and are indicated by the shaded 
areas on Table 1-9. 

As derived from Table 1-9, five of the seven unique characteristics at Level I appear as 
areas of concern. In rank order from most to least problematic, Level I programs tend not 
to: 

■ provide a coherent sequence of instruction required for all vocational technological 
programs (Organization #4) 

■ serve 100% of the students in the targeted grades (Organization #2) 

■ provide an experience equal to one year in length or equivalent to 225 minutes per 
week (Organization #3) or utilize ADE-approved curriculum (Curriculum #8) 

■ provide an exploration experience for all occupations (Organization #1). 



At Levels II and III, only one characteristic is unmet by at least half of the programs. Both 
of the higher levels tend to not have students previously enrolled in a Level I experience. 
This "prerequisite" characteristic is the most frequently unmet "unique" characteristic for 
all programs across all levels. 

Similar to the findings for shared characteristics, the 12 programs were more successful in 
satisfying unique characteristics for Level III than for Level II, and more successful in 
satisfying characteristics for Level II than for Level I. 

In analyzing both Table 1-8 and 1-9, it is somewhat arbitrary to designate a characteristic 
as indicating a problem if less than half of the programs have that characteristic. Morrison 
Institute analysts could just have easily used two-thirds or three-quarters of sites without a 
characteristic as a criterion measure. In either of the latter cases, there would be 
significantly more "problem areas" notod in Tables 1-8 and 1-9. ADE/VTE personnel should 
review the findings with this caveat— and their own criteria — in mind. 
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DISCUSSION 



The two previously discussed studies were conducted to determine the extent to which 
the VTE Model is being implemented in Arizona schools. In both studies, "modelness" was 
defined on the basis of ADE-defined criteria as outlined in the Arizona Model for Vocational 
Technological Education Program Characteristics Checklist. Morrison Institute researchers 
investigated the extent to which individual levels of the VTE Model were being 
implemented, as well as the extent to which comprehensive Model programs (i.e., those 
with a Level l-ll-lll sequence) were being implemented. 

Implementation Study 1 involved surveying 172 schools in the state with formula-funded, 
state-approved VTE programs. Forty-five percent of these schools (72) believe they offer a 
Level l-ll-lll comprehensive VTE program in at least one occupational/program area. A total 
of 145 schools beiieve they offer VTE Model programs in at least one of the levels of 
curriculum defined by the VTE Model. 

A total of 1 00 districts were surveyed in Implementation Study 1 . Fifty-two of these 
districts reported having at least one comprehensive VTE Model program. Of these 52 
districts, 29 districts had received "Model site" funding to assist them ; n implementing a 
comprehensive program. 

Implementation Study 1 raised several questions about the implementation of the VTE 
Model in Arizona. Moreover, there was some question about the veracity of self-reported 
data. Both as a "reality check" on the findings from Study 1, and as a more in-depth pilot 
study about the implementation of the VTE Model, Implementation Study 2 was designed. 

Implementation Study 2 was conducted to determine the extent to which the Arizona 
Vocational Technological Education Mode! was being implemented in a sample of 1 2 
schools and to identify problems with its implementation. The data for the 1 2 schools to 
which site visits were made revealed that none of the programs had all of the 
characteristics of a fully implemented Model program as defined by ADE. However, the 
results varied considerably by program level, with programs having progressively more 
Model characteristics as the program level increased from Level I to Level II to Level III. 

The site visits revealed several reasons that Level I courses generally did not satisfy the 
Model characteristics for this level. Most Level I courses did not provide skill exploration 
across all or most occupational areas, but instead covered a limited number of 
occupational areas. Most Lev ! I courses were only a semester in length, rather than a year 
as called for in the VTE Model Characteristics. A Level I course typically was not required 
of all students at the targeted grade level, and a Level I course seldom was a prerequisite 
for courses at Levels II and III. Because of the survey nature of Level I courses, they 
normally were not industry realistic with regard to either instructional activities or facilities 
and equipment. 

One reason that Level I courses were not better integrated into an instructional sequence is 
their location in different schools from the Level II and Level III courses. The site visits 
revealed that vocational-technological courses and programs typically are not articulated 
well between high schools and their feeder schools. The junior high schools and high 
schools normally operated quite independently, and it was unusual for programs to be 



ERLC 



24 



33 



planned well across these levels. Thus, even when junior high school courses satisfied 
many other model characteristics, they rare!y were designated as prerequisites for the high 
school courses. Furthermore, it seldom is feasible for a district to purchase an array of 
industry-realistic equipment for both junior and senior high schools, so districts that can 
afford such equipment typically locate it in the high schools to serve their need for more 
advanced training. 

Level II courses had more Model program characteristics than Level I courses at all 12 of 
the visited sites. Level II courses were better articulated with Level III courses and often 
shared some of the advanced equipment. Level II courses were much more successful than 
Level I courses, but not as successful as Level III, in satisfying Model characteristics 
related to industry realism of instructional activities and of facilities and equipment. In 
addition, courses designated as Level II in several schools did not satisfy the Model 
characteristic of providing skills development for a cluster of related occupations, but 
instead were the first course in a two-course sequence in a single occupational area (e.g., 
Video Production I and Video Production II). 

Level III courses were most successful in satisfying the Model program characteristics. 
Four of the 12 programs had only one rating of "no" at Level III, and seven more had five 
or fewer "no" ratings at this level. However, even at this level, certain Model 
characteristics were not present at several sites. Students were required to complete both 
Level I and Level II courses prior to Level III at only three of the 12 sites primarily because 
a Level I course was not a requirement at most sites. The curriculum included all six of the 
Model strands at only two sites at Level III and also at Levels I and II. Personnel at most 
sites either were not knowledgeable about the Model strands or planned and implemented 
their programs prior to knowing about them. Finally, several Level III programs were 
evaluated as not having instructional activities, facilities, and equipment that met or 
simulated realistic industry standards. 

The 12 programs consistently satisfied several of the Model characteristics at all three 
program levels. Characteristics for which there were few "no" ratings at any level related 
to the experience of having administrative support, the teacher serving as a facilitator of 
instruction, students working in groups or teams frequently, and teachers having recent 
inservice training. It seems virtually inevitable that courses would receive "yes" ratings on 
at least the first three of these characteristics because of how they are worded. A school 
probably could not offer a course if the course did not have administrative support, 
teachers generally serve as instructional facilitators who manage a variety of learning 
experiences, and teachers almost invariably report that their students work in groups or 
teams frequently. Rewording these characteristics could help to make them more 
discriminating between stronger and weaker programs; eliminating them is also a 
possibility. 

Although no individual program had all of the Model program characteristics at all three 
levels, one program had nearly all of them (57 of 61 across these levels) and two others 
had only one or two "no" ratings across Levels II and III combined. All three of these 
programs had at least one "no" rating because a Level I experience was not a prerequisite 
for taking higher level courses. 
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The site visit data on comprehensive and non-comprehensive programs, as classified by 
local district officials who completed the VTE Model Components Questionnaire in Study 
1, indicated that self-classified comprehensive and non-comprehensive programs differ 
from each other primarily at Level I. At Level I, non-comprehensive programs either do not 
offer a Level I experience or do not offer an experience that meets ADE criteria. At both 
Level II and Level III, however, the overall site visit ratings were similar for comprehensive 
programs and non-comprehensive programs. 

Data suggest that, in most cases, the district officials who classified their programs as 
comprehensive or non-comprehensive programs did so at least partly on the basis of their 
overall evaluation of the programs rather than on a careful audit of each program against 
the VTE Model Program Characteristics Checklist. This finding suggests caution in 
interpreting the results of Implementation Study 1. Even so, the comparison of 
comprehensive and non-comprehensive sites shows that sites that report themselves as 
comprehensive do tend to meet more of the ADE-defined criteria for a comprehensive 
Model program. This lends some credibility to the self-reported data gathered in 
Implementation Study 1. 

Site visit data on Model-funded and nonfunded programs indicated that funded programs 
meet more of the ADE Program Characteristics than do nonfunded programs at all levels of 
Model implementation. Funded programs are more "in compliance" with the program 
characteristics with respect to ADE curricula, Model strands, and overall alignment. This 
could be due to the fact that ADE staff might be more visible and provide more technical 
assistance to sites that receive funding. Furthermore, Model-funded sites appear to have a 
distinct advantage over nonfunded sites in terms of more and newer facilities and 
equipment that meet industry standards. In sum, ADE "Model site" funding appears to 
make a difference in the degree to which sites are implementing the VTE Model. 

A limitation of the site visit study is that it included only 12 programs. Certainly, 
considerable caution should be exercised in making assumptions about the degree to 
which the specific findings from the sample in this study would be represented in the 
entire population of ADE-approved VTE programs in Arizona. Nevertheless, it is likely that 
the data from this pilot study generally are good indicators of the Model program 
characteristics that programs typically satisfy and those they often do not. 
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PART TWO 



LEA PROCEDURES IN CONDUCTING 
PERFORMANCE STANDARDS EVALUA TIONS IN ARIZONA 



This section of the report describes Arizona's Performance Standards and Measures and a 
study conducted by Morrison Institute analysts of how local educational agencies conduct 
their Performance Standards evaluations. 



ARIZONA'S PERFORMANCE STANDARDS 

In accordance with the Perkins Act of 1990, ADE's Division of Vocational Education 
developed "Performance Standards" (Figure 2) to measure student outcomes with 
reference to occupational training programs. The implementation of Performance Standards 
evaluations reflected a shift away from compliance reporting to reporting on student 
performance. As stated in the ADE/VTE document A Teacher's Guide to Performance 
Standards for Vocational Technological Education (henceforth the Teacher's Guide), "The 
philosophy behind Performance Standards is to establish goals to strive for through 
program improvement" (p. 1). 

ADE/VTE developed a comprehensive manual and the Teacher's Guide to assist LEAs in 
implementing Performance Standards evaluations. On the Performance Standards reporting 
forms applicable for 1992-93 and 1993-94, LEAs were asked to provide student 
information in four categories: 

■ competency gains in the achievement of basic and more advanced academic skills; 

■ occupational competency attainment; 

■ high school enrollment/graduation; and 

■ postsecondary training or education, military service or employment. 5 



Performance Standards data are to be collected annually and compiled by local districts. 
Reporting forms are extensive and request information on each standard (36 questions 
total) for each course in each program. Most questions require local districts to 
disaggregate data for each course/program by the number of regular, special needs, and 
total students (i.e., there are three numbers reported in each "cell" of the reporting form). 
Forms require the reporting of numbers and percentages of students judged to have 
demonstrated academic skills, attained occupational competencies, and so forth. Forms are 
comprised exclusively of dichotomous (yes-no) and categorical data. 



5 Districts/schools were exempt from reporting on this measure for FY 1992-93. 
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ADE/VTE documents enumerate guidelines for districts/schools in preparing the 
Performance Standards evaluation reports. These include the appointment of a Local 
Evaluation Coordinator and Local Evaluation Team, who are charged with conducting the 
evaluation and ensuring compliance with data gathering methods and reporting procedures. 

Figure 2. Arizona's Performance Standards 



MEASURE 1 . Demonstrate competency gains, including student progress, in the achievement of basic and 
more advanced academic skills. 


STANDARD 1.1: 


All of the course completers who do not possess the identified skills at the 
beginning of the course/program will demonstrate gains in the achievement of the 
related basic academic skills associated with the course/program. The student's 
course/program may be modified through an Individual Vocational Education Plan 
(IVEP). 


STANDARD 1.2: 


All of the course completers who do not possess the identified skills at the 
beginning of the course/program will demonstrate gains in the achievement of the 
more advanced academic skills associated with the course/program. The student's 
course/program may be modified through an IVEP. 


MEASURE 2. Demonstrate occupational competency attainment. 


STANDARD 2.1: 


Eighty percent {80%} of the course completers will demonstrate attainment of at 
least eighty percent {80%} of the occupational competencies/tasks associated with 
the course. The student's course/program may be modified through an IVEP. 


STANDARD 2.2: 


One hundred percent <100%} of the program completers will demonstrate 
attainment of at least eighty percent (80%) of the occupational competencies/tasks 
associated with the course. The student's course/program may be modified through 
an IVEP. 


MEASURE 3. Continue attending or complete secondary school. 


STANDARD 3.1: 


Ninety percent (90%) of the students who enroll in a course which is part of an 
approved vocational technological program will continue attending/complete 
secondary school. 


MEASURE 4. Placement into additional training or education, military service/ or employment. 


STANDARD 4.1: 


Ninety percent (90%) of the students completing an approved 
vocational/technological program will be placed into additional trainr g or education 
in an accredited private or public postsecondary institution, military service, or 
employment. 

Approved by the Arizona State Board for 
Vocational Technological Education 

April 27, 1992 
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PERFORMANCE STANDARDS PROCEDURES SURVEY 



Background 

Beginning with the FY 1992-93 school year, districts/schools were to complete 
Performance Standards reports, due to ADE/VTE in September 1993. District-appointed 
Local Evaluation Coordinators were to submit local program improvement plans intended to 
address any weaknesses indicated by their 1992-93 performance standards data. 
ADE/VTE personnel planned to utilize aggregated Performance Standards data to assess 
VTE program outcomes statewide and target areas for program improvement- 

The current research project was commissioned to examine local procedures used in 
preparing Performance Standards reports, with the ultimate goal to assess issues of 
reliability and validity as they pertain to the state's database. These issues are relevant to 
the utilization of Performance Standards data for purposes of program evaluation. 

As of November 1994, ADE/VTE personnel had documented a number of problems in 
district-reported Performance Standards data. According to ADE documentation provided 
to local vocational directors at a meeting held on November 13, 1993, problems included 
the following: 

Regarding the reporting of basic and advanced academic skills— 

Some courses and/or programs were not included in the assessment process. 

The same data were reported for the attainment of both basic and advanced academic 
skills. 

Assessment procedures only pertained to one term of the academic year. 
Information was not reported for basic and/or advanced skills. 
Regarding the reporting of occupational competencies — 
Same problems as for academic skills, plus... 

Data on competency attainment were based on a course grade rather than on the 
actual measurement and recording of competencies. 

Additionally, ADE noted that many districts did not provide appropriate statements of 
assurance and local improvement plans. Some districts also did not measure/report the 
performance of special needs students or have the required representation on the local 
evaluation team (i.e., the group charged with implementing the Performance Standards 
evaluation). 

Drawing upon the above information, ADE/VTE personnel expertise, and the ADE 
Teacher's Guide (1992 and 1993 versions), Morrison Institute researchers designed a 
survey to assess local procedures in conducting Performance Standards evaluations. The 



survey was comprised of seven sections and was intended to assess local procedures with 
respect to measuring: basic and advanced skills (Section I), occupational competencies 
(Section II), continued enrollment (Section III), and the performance of special needs 
students (Section IV). Section V requested information pertinent to preparing and 
conducting the local evaluation, including information about the local evaluation team, the 
amount of training and inservice received by those implementing the evaluation, and 
procedures pertaining to the filing of a "minority report." Section VI was designed as a 
user evaluation of the state's Performance Standards system. All questions in Sections I 
through VII were multiple choice to facilitate completion of the survey. Most items 
contained an "Other" response to allow for open-ended responses. Finally, Section VII 
allowed for respondents to offer additional comments regarding any aspect of the survey 
or Performance Standards system. 

Sampling and Methods 

A cross-sectional sampling design was selected to represent programs statewide in each of 
the six primary occupational areas (Agriculture, Business, Health Occupations, Marketing, 
Occupational Home Economics, and Trade and Industry). A sample size of 30 per program 
area was selected as appropriate for this type of exploratory study. 6 Occupational area 
samples were stratified by program type. For example, within the Agriculture area, the 
sample was comprised of programs representing Agriculture Business/Management, 
Agriculture Mechanics, Horticulture, and Renewable Natural Resources in proportion to 
their representation in the total number of state-approved Agriculture programs. 

An additional parameter for sample selection was that no one local evaluation coordinator 
(a district representative) would receive more than two surveys. In the cases where two 
surveys were sent to the same local evaluation coordinator, efforts were made to target 
the surveys to two different schools/programs in the district. This sampling parameter 
limited the number of health occupations programs that could be surveyed without 
duplicating districts/schools. A total of 14 unique districts and schools were identified in 
the Health Occupations area; all were surveyed. 

In sum, the final sample was comprised of 30 Agriculture Education programs, 30 
Business Education programs, 14 Health Occupations Education programs, 30 Marketing 
Education programs, 30 Occupational Home Economics Education programs, and 30 Trade 
and Industry programs for a total of 164 programs. These 164 programs were distributed 
across all 15 Arizona counties in 92 districts and 108 schools. This sample was the most 
inclusive cross-sectional sample possible, given the two-survey per district limit imposed 
by researchers. 

Surveys were customized by printing the district, school, occupational area, and program 
area targeted for study on the survey. Surveys asked for the name and position of the 
person completing the survey and their telephone number for purposes of follow-up. The 
confidentiality of site-specific responses was guaranteed. 



Following Isaac, S. and Michael, W.B. (1987). Handbook in Research and Evalual >n. San Diego, CA: EdITS 
Publishers. 
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The survey was distributed by mail in February 1994 with a one-month turnaround time. 
Extensive follow-up was conducted during April and May through repeated mailings and 
telephone calls. Follow-up was conducted both to obtain responses as well as to clarify 
information on returned surveys. 

Survey Response Rates 

A total of 77 of the 92 districts surveyed responded for a district response rate of 84 
percent. Ninety-two of the 108 schools surveyed responded for a school response rate of 
85 percent. Several district/school responses indicated that the school no longer offered 
the program targeted on the survey. These responses were counted in the district/school 
response rate, but surveys were considered unusable. Other unusable surveys were those 
that were returned with a majority of items left blank. Response rates by occupational 
area, and the total number of surveys used in the final analysis, are provided in Table 2-1 . 



Table 2.1 

VTE Performance Standards Survey Response Rates by Occupational Area 



Occupational 
Area 


# Programs 
Surveyed 


# Responses 


Overall 
Response 
Rate 


# Usable 
Surveys 


Percent 
Usable 
Responses 


Agriculture 


30 


25 


83.3% 


24 


80.0% 


Business 


30 


25 


83.3% 


25 


83.3% 


Health Occupations 


14 


12 


85.7% 


1 2 


85.7% 


Marketing 


30 


26 


86.7% 


24 


80.0% 


Occ. Home Economics 


30 


25 


83.3% 


23 


76.7% 


Trade & Industry 


30- • 


23 


76.7% 


20 


66.7% 


Totals 


164 


136 


82.9% 


128 


78.0% 



Survey Results 

All results reported in this section reflect the total number of usable surveys analyzed (N = 
128). Key findings are summarized by section. Specific results for each question are 
contained in Appendix D. Results by occupational area are available as separate 
documents. 

Section I: Measuring Basic and Advanced Academic Skills 

Morrison Institute's Performance Standards survey included nine questions each about the 
measurement of basic and advanced academic skills. Questions were designed to 
determine what academic skills are assessed and how they are assessed in order to allow 
researchers to determine whether or not methods are "reasonable and reliable" as per ADE 
directives. 
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Four questions pertained to the identification of basic and advanced academic skills. 
Survey recipients were first asked to indicate whether they used the same or different 
formal procedures to identify academic, skills in all courses in their program. A majority of 
the 128 respondents indicated that they identified both basic and advanced academic skills 
in all courses using the same procedure. A total of 100 respondents (78 percent) indicated 
uniformity in procedures to identify basic skills; 96 (75 percent) indicated this uniformity to 
identify advanced skills. Conversely, nearly one-fourth of all respondents for both basic 
and advanced academic skills said they used different procedures or did not formally 
identify academic skills. 

Survey recipients then were asked to indicate the procedure used to identify basic 
academic skills related to the occupational program in question. A variety of procedures 
were indicated, including the use of teacher judgment, the analysis of Arizona's Essential 
Skills curriculum frameworks, the analysis of tests (commercial and district-developed), 
and the analysis of either the VTE or academic curriculum. The most frequent response 
indicated that LEAs rely on teacher judgment to identify the basic and advanced academic 
skills associated with VTE courses. Thirty-two percent of the respondents indicated 
teacher judgment is used to identify basic skills; thirty-four percent rely on teacher 
judgment to identify advanced academic skills. Less than one in five districts indicate using 
the Essential Skills frameworks to identify academic competencies related to their courses. 

Asked to indicate what specific academic skills are identified and for which students are 
assessed, most respondents said that arithmetic/mathematical skills are identified and 
assessed (87 percent for basic; 84 percent for advanced) as well as communication skills, 
including reading and writing (86 percent for basic; 84 percent for advanced). 
Approximately half of all respondents indicated assessing both basic and advanced science 
skills. Four percent of the respondents indicated assessing both basic and advanced 
"language skills." 

Finally, survey respondents were asked to indicate who is primarily involved in the process 
to identify the specific basic academic skills associated with the courses in their particular 
program. Over half of the respondents noted that individual VTE course instructors are 
primarily involved in the process to identify both basic and advanced academic skills. 
Twelve percent indicated that exclusively VTE staff (collectively) are involved in the 
identification of academic skills. In very few cases (less than ten percent) are academic 
staff involved in identifying academic skills relevant to VTE courses. 

The remaining five questions in Section I examined procedures to assess students' 
academic skills. Respondents first were asked whether the same assessment methods are 
used in every course. Ninety percent of the respondents said "yes" in relation to assessing 
basic skills; 88 percent said yes in relation to advanced skills. Respondents then were 
asked whether a test is used in every course. Over ninety percent indicated that tests are 
used. Respondents were asked to indicate the kind of test used. Over fifty percent 
indicated the use of a teacher-developed test to assess both basic and advanced skills. A 
variety of other test instruments account for the remaining responses. Of these other test 
instruments, 13 percent of the respondents indicate using the Arizona Student 
Assessment Program (ASAP) instruments to examine basic skills; 1 1 percent said they use 
ASAP to assess advanced academic skills. 
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If respondents indicated using a test to measure academic gains, they were asked whether 
or not all students in every course are pre-post tested. Eighty-five percent of the 1 28 
respondents said they pre-posttest to assess basic skills; 82 percent said pre-posttesting is 
used to assess advanced academic skills. 

If respondents indicated pre-posttesting, they also were asked to define "gain" relative to a 
difference in pretest and posttest scores. Seventy-one people responded to this item with 
respect to both basic and academic skills. A majority of respondents indicated that a one 
percent or one answer difference was considered as a gain. Several people indicated that 
the same score on the pretest and posttest was considered a gain, while several others 
indicated they were still "working on" a definition of gain. Only six of the 71 respondents 
indicated that students must score at least five percent higher on the posttest than on the 
pretest as evidence of a gain in basic skills. These six respondents represented five of six 
occupational areas (all except Occupational Home Economics). 

Finally, respondents were asked to note other methods, besides tests, used to determine 
academic gain and to define "gain" with respect to other measures. For both basic and 
advanced academic skills, thirty-one respondents indicated using a measure of gain other 
than pre-posttest scores. A majority of these respondents indicated the use of formal and 
informal teacher observations of student work (e.g., project completion, lab performance, 
unit quizzes, portfolios). Sample definitions of gain are as follows: 

■ Gain is defined through problems, projects, and lab work. 

■ [Gain is the] mastery of skills NOT passed on the pretest. 

■ Gain is defined as improvement or acquisition. 

■ [Gain is] successfully completing courses within the program. 

■ Gain is defined through completion of class projects. 

■ Gain is the ability of the student to demonstrate an elevated usage of academic skills 
needed to succeed in the occupational area. 

One respondent was more specific regarding teacher assessments, writing the following: 

"A list of Essential Skills is used to evaluate students before and after each course. All 
skills that are attained are checked off and academic gain is determined by how many 
new skills can be checked off at the end of a course that were not previously attained. 
Gain is defined as competence and understanding of a specific skill." 

Only one respondent indicated measuring gain in a different manner than through tests or 
teacher observations/assessments of student work. In this case, gain is determined on the 
basis of a student's overall grade point average (i.e., "GPA progress"). 



Section II: Measuring Occupational Competency 

The Performance Standards survey included 12 questions about the measurement of 
occupational competencies. Questions were designed pertaining to the identification of 
occupational competencies, the measurement of these competencies, and definitions 
relevant to course and program completion. 

Survey respondents were asked to indicate how occupational competencies/tasks were 
identified for their particular program. Most respondents (1 1 1 , or 87 percent) said 
competencies/tasks were identified for all courses using the same procedure. The 
remainder indicated the use of different procedures or said that competencies are not 
formally identified. 

Survey recipients then were asked to indicate the procedure used to identify occupational 
competencies/tasks for the courses in their particular program. Over half of the 128 
respondents (51 percent) indicated using ADE-validated competency/task lists. An 
additional 19 percent said that the VTE program curriculum was analyzed, while another 
17 percent said that teacher judgement was used. Other responses indicated a 
combination of approaches. 

Respondents also were asked to indicate who was primarily involved in the process to 
identify specific occupational competencies and tasks associated with the courses in their 
particular program. Fifty-eight respondents (45 percent) said the VTE course instructor(s) 
are primarily involved. Others indicated that no personnel were involved, per se, but rather 
that they relied on ADE competency/task lists. Others involved included local advisory 
councils (12 percent), school/district VTE teams (5 percent), and combinations of the 
above (9 percent). Only four respondents indicated the specific use of local/state/federal 
business and industry representatives as those primarily involved in the process to identify 
occupational competencies. 

Regarding student assessment methods, survey respondents were asked whether or not 
they used similar methods in every course to assess student attainment of occupational 
competencies and tasks. Eighty-eight percent of the respondents indicated "yes," similar 
methods are used. 

Also regarding student assessment methods, survey recipients were asked whether or not 
they used a course-based performance measure of occupational competencies (as, for 
example, a project or skill demonstration) in every course. Again, 88 percent responded in 
the affirmative. Finally, with respect to student assessment methods, survey respondents 
were asked to indicate any and all other methods used to determine the attainment of 
occupational competencies and tasks. Responses varied. Not including nonrespondents: 

■ 125 (98 percent) said they used teacher observation and judgement of student performance. 

■ 113 (88 percent) used teacher-developed paper and pencil tests. 

■ 52 (41 percent) said they used on-the-job supervisor evaluations. 

■ 47 (37 percent) indicated they used student evaluations. 

■ 1 1 (9 percent) indicated they used other methods, such as ADE-developed "pencil and paper" tests. 
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Section II sought additional information about definitions of VTE "programs" and 
"courses." A majority of respondents (64 percent) said that their program is comprised of 
one to three courses, each course being a year in length. 

ADE defines a course completer as "a student who is enrolled within the first grading 
period and remains enrolled through the end of a course" (Teacher's Guide, p. 16). Survey 
respondents were asked to indicate how they define a course completer. The most 
frequent responses are as follows: 

■ 69 (54 percent) said only students who receive credit for the course. 

■ 45 (35 percent) said all students enrolled in the first grading period who finish the course. [Note: This 
is the ADE definition.] 

■ 1 1 (9 percent) gave other definitions such as a student who completes 80 percent or more of the 
competencies. 

Regarding the requirements of their program, survey respondents were asked how many 
courses at a minimum, must a program completer have successfully passed. Most 
respondents (81 percent) indicated that a program completer is one who passes one to 
three (1-3) three courses. 

In reference to the requirements of their program, survey respondents were asked whether 
or not they required program completers to obtain applied and occupational experience in 
the industry for which they are preparing es part of the program. Roughly half of the 
respondents (52 percent) said "no," participants were not required to obtain practical 
experience. 

For the 58 respondents (45 percent) who stated that applied/occupational experience is a 
program requirement, they were asked the types of applied experiences required. Their 
responses are as follows: 

■ 49 (38 percent of the total sample) said school-based work simulation(s). 

■ 30 (23 percent of the total sample) said clinical experience, paid or unpaid. 

■ 22 (17 percent of the total sample) indicated cooperative education. 

■ 6(5 percent of the total sample) indicated apprenticeship program(s). 

■ 6 (5 percent of the total sample) indicated other requirements such as a Supervised Occupational 
Experience (SOE) program 
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Section III: Measuring Continued Enrollment 



Survey recipients were asked five questions about enrollment and efforts to keep students 
in school. Survey recipients first were asked to identify the definition of "course 
enrollment" for their particular program. Of the 128 respondents: 

■ 60 (47 percent) defined course enrollment as all students enrolled In the course throughout the term. 

■ 34 (27 percent) defined course enrollment as all students who register for a course within the first 
grading period. 

■ 30 (23 percent) defined course enrollment as all students who successfully complete course 
requirements. 

■ 3 (2 percent) described course enrollment as "most of the students who register within the first 
grading period and stay the whole year are course completers." 



1 (>1 percent did not respond. 



Survey respondents also were asked whether their school has a management information 
system (MIS) that allows them to track students who leave the course or program. 
Approximately half (49 percent) said that they do have an MIS system; the other half (49 
percent) said they do not have an MIS system. Two percent did not respond. 

If respondents indicated that they do not have an MIS system, they were asked how they 
track students. Responses invariably mentioned that teachers are responsible for tracking 
students, with records compared against .counselor or other school files. Follow-up studies 
are typically conducted by program staff through mail and telephone contacts. 

Several respondents said they were unsure of how students were tracked; still others 
wrote that tracking was conducted at the district level. In one case, a university conducts 
follow-up; in another, an alumni association conducts the follow-up studies. One 
respondent said follow-up is by "word-of-mouth" and student visits. Another admitted that 
students are not tracked at all. Invariably, difficulties of tracking students "the old- 
fashioned way by paper and pencil" were related to the size of the school and student 
bcdy. 

Survey respondents were asked the degree of emphasis placed on providing teachers with 
inservice or training that addresses reasons why students drop out of school. Of the 128 
respondents, 123 (96 percent) said that this was a moderate to low priority with some, 
few, or no activities. Respondents also were asked if the emphasis (from Question 3.3) is 
appropriate. Sixty-four respondents believed the emphasis to be appropriate; 63 
respondents believed the emphasis was not appropriate. (One did not respond.) 

Finally, survey recipients were asked how effective they considered their program to be in 
keeping students enrolled in school. Half (50 percent) believed their program to be "very" 
effective; the remaining half (50 percent) believed their program to be only "somewhat" or 
"not very" effective. 
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Section IV: Procedures and Attitudes Related to Special Needs Students 

The Performance Standards evaluations require that VTE programs separately report on 
special needs students (i.e., those requiring an Individualized Vocational Education Plan, or 
IVEP). Separate reports are required by course and program for each standard measured. 
Eight questions on the survey related to procedures and attitudes applicable to educating 
and assessing special needs students. 

First, survey recipients were asked the extent to which they believed that special 
populations were represented in their program. Of the 128 respondents: 

■ 92 (72 percent) believed that special needs students are proportionately represented in their program. 

■ 31 (24 percent) indicated that special needs students are overrepresented; that there are too many in 
the program, 

■ 5 (4 percent) said special needs students are underrepresented; that there are too few in the program. 

Respondents were asked to indicate the extent to which instructors in their programs were 
involved in developing iVEPs. Eighty-two respondents (64 percent) said instructors were 
"often" involved; 43 (37 percent) said instructors were "sometimes" or "rarely" involved. 
Survey respondents also were asked the extent to which IVEPs were used by instructors 
in their particular program. Thirty-eight percent said that IVEPs were "often" used; 61 
percent indicated they were "sometimes" or "rarely" used. 

Additionally, survey recipients were asked the extent to which instructors in their program 
coordinated instruction with other instructors working with IVEP students. Twenty-six 
respondents (20 percent) said there was "frequent" coordination; 98 (77 percent) said 
there was "some" or "infrequent" coordination. 

According to the ADE's Performance Standards, IVEP students are expected to perform as 
well as regular students. Survey respondents were asked, in their opinion, how many 
instructors in their program believed that this is a realistic expectation. Of the 1 28 
respondents, 22 (17 percent) said "all" instructors believed this is a realistic expectation. 
An additional one-third of the respondents (34 percent) said "most" teachers believed this 
is realistic. Fifty-six respondents (44 percent) indicated that a majority of their teachers do 
not believe that it is realistic for IVEP students to perform as well as regular students. 

Regarding the assessment of IVEP students, respondents were asked if basic and 
advanced academic skills for IVEP students were measured in the same way as for regular 
students. Most respondents (87 percent regarding basic skills; 88 percent regarding 
advanced skills) said that IVEP students were assessed in the same way. 

Similarly, respondents were asked if the attainment of occupational competency for IVEP 
students was measured in the same way as for regular students. Over 100 respondents 
(85 percent) said that IVEP and regular students were assessed in the same way. 
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For respondents who responded "No" to any of the assessment questions, they were 
asked how they modified assessment methods for 1VEP students. Some responses simply 
indicated that assessment methods were "modified as appropriate/' sometimes based on 
the input of a special education teacher. Specific modifications included the following: 

■ IVEP students are given more time to acquire the same skills/competencies, and/or allowed more time 
to complete tests/tasks. 

■ Fewer competencies are required; some are "deleted." 

■ Job skill requirements are modified. 

■ A lower standard of success is used for IVEP students (e.g., 50% or 70% attainment of skill mastery 
versus 80% for regular students). 

■ IVEP students may be allowed to have assistance in understanding/taking the test 
(e.g., assisted with the reading of questions). 

■ "More individual observation by the teacher is performed." 



Section V: Preparing and Conducting the Local Evaluation 

In this section of the survey, ten questions were asked to obtain general information about 
the people and procedures used to complete Performance Standards evaluations. Survey 
recipients were asked if the district's VTE director acted as the local evaluation 
coordinator. A majority (1 10 or 86 percent) responded "yes" to this question. 

Regarding student assessment and Performance Standards data collection, respondents 
were asked to indicate any and all types of training and support that their district provided 
for VTE teachers. Of the 128 respondents: 

■ 93 (73 percent) identified district inservice training. 

■ 74 (58 percent) indicated district support such as administrative and clerical assistance. 

■ 39 (30 percent) indicated state-sponsored training. 

■ 35 (27 percent) identified ADE staff assistance. 

■ 7 (5 percent) did not respond. 



Survey respondents also were asked how often evaluation information was shared with 
the teachers for their particular program. A little over half (55 percent) indicated that 
information was regularly shared. The remaining 45 percent said information was shared 
only occasionally, seldom, or never. 

Survey respondents were asked if there was a local evaluation team for their program. 
Over 100 respondents (1 12 or 88 percent) indicated "yes." Recipients also were asked 
how many members were on the local evaluation team for their program. Of the 1 12 
respondents who indicated they did have a team, team membership ranged from a low of 
2 to a high of 20. Reportedly, the "average team" has 6 members. Asked about the 
composition of the evaluation teams (as per ADE requirements), all teams had a teacher 
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representative, 93 percent had a business and industry representative, and 83 percent had 
a representative for the special needs population. 

Asked if the local evaluation team had received training for its role in conducting 
Performance Standards evaluations, 90 respondents (70 percent) said "yes." The 
remaining respondents either said "no" or were unsure (or did not respond). Additionally, 
survey recipients were asked to briefly describe the nature of the training provided. 
Responses varied. Of the 90 respondents who indicated that they had received training, 
69 described the nature of the training received. Of these 69 respondents: 

■ Forty-two respondent (61 percent) indicated that training was "limited" ranging from brief (e.g., 30 
minutes or less) "overview" explanations to one-hour presentations. One respondent described such a 
meeting as n a formality. " 

■ Fifteen respondents (22 percent) indicated training sessions between two and five hours. 

■ Five respondents (7 percent) indicated two days of training provided through a Rural Technical 
Assistance Project. 

■ Two additional respondents (3 percent) reported receiving three days of inservice with their local 
director. 

» Five respondents (7 percent) indicated "ongoing" training provided throughout the year. 



Regarding various roles of the local evaluation team, respondents were asked if, in the 
case that Performance Standards were not met, whether the team played a key role in 
developing a local improvement plan. Eighty-eight respondents (69 percent) said that the 
team did, in fact, play a key role. Survey respondents also were asked if it was clearly 
understood that team members who disagreed with the district report could file a minority 
report. Over 100 (103 or 80 percent) said that this was clearly understood; furthermore, 
nine respondents (7 percent) indicated filing a minority report. 

Section VI: User Evaluation of the Performance Standards System 

This section of the survey was designed to determine local VTE personnel opinions about 
the usefulness, accuracy, and importance of Arizona's Performance Standards and 
Measures system. For each of sixteen items, survey recipients were asked to indicate their 
agreement or disagreement with an item on a four-point scale. Based on the distribution of 
responses across the scale, mean values were calculated for each item. An overall mean 
was also calculated (i.e., a mean of the 16 item means). Items and mean values are 
provided in Table 2-2 on the following page. Frequency distributions for each item are 
provided in Appendix D. 
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Table 2.2 



User Evaluation of Performance Standards System (N = 128) 



Item 



Mean Rating * 



6.1 The Performance Standards requiring that students demonstrate gains in basic and 
advanced academic skills are important for evaluating VTE programs in my school/district. 

6.2 My school/district can accurately determine whether each student in a VTE course 
makes achievement gains in basic and advanced academic skills during a school year. 

6.3 The Performance Standards requiring that students attain occupational competency are 
important for evaluating VTE programs in my school/district. 

6.4 My school/district can accurately determine whether course completers attain 80% of 
the occupational competencies for each course. 

6.5 My school/district can accurately determine whether program completers attain 80% the 
occupational competencies for each program. 

school i$ itnptjtxmi for evaluating VTE programs m my JchteU&tttcL " * : 7?llll 

6.7 My school/district can accurately determine whether 90% of the students in each VTE 
course stay enrolled in or complete high school. 

6.8 Separately ^tracking wjuiar and iVEP students is important for evaluate VT£ programs •■ ■ ■ _ • 

6.9 My school/district car. accurately identify the number of both regular and IVEP students 
in each VTE program. 

6.10 Local evaluation teams play an important role in my school/district's VTE program. 

6/11 My district would have a Jccsi evaluation team for each S/§M 
were no state involvement in the programs, 

6.12 It Is reasonable to expect school district to have a local evaluation team with a teacher 
representative, a business and industry representative, and special populations representative 
for each VTE program. 



2.85 
2.88 
3.15 
3.10 
3.05 

3.19 



3.27 
2.95 




6. 13 it in teasoruihte to expect school districts to maintain mti mbmit the Performance 
Standards reports resumed by federal feg&HtixMi and ADE. 

6.14 Once the "bugs* ar* wo'fJced ot*t, the Arizona VT£ standards will be- useful lot waking 
important rm$fov$mertt$ m my schaoi/dtetrict VT£ proopmn. 

6.15 The costs fce., time, monay, etc) of keeping the stst^requked student p&rforrnance ' 

: be ju$ti fled 'Sy; ; the • benefits ta : rr)y ^oar/di^rftt^^^^ 

6V1 6 i would recommend to school. districts in other states- that they use a VTE performance 
standards iyiitHrr) like Airt*tma's. ■ • : : -. ; : " '[■:■' • • . " • 



2.88 

if IS 



# For purposes of interpretation, the overaK mean value for ail 16 items was computed 
(2.78.) Items with a mean score below 2.78 indicate relative disagreement based on the 
overall mean. These items are shaded in tho table. 



4 = Strongly 

Agree 
3 - Agree 
2 = Disagree 
1 ss Strongly 
Disagree 
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Section Vli: Additional Comments 



Finally, Section VII of the survey requested additional comments regarding any aspect of 
the Performance Standards system. Most respondents did not write additional comments. 
Those who did write comments indicated a need for more time to do Performance 
Standards record keeping or reporting, and/or dissatisfaction with the Performance 
Standards system. Comments are reproduced verbatim (except for minor editing for 
spelling, grammar, and clarity) in Appendix D. This section is illustrative of the comments 
provided. 

Time-Related Comments 

■ Instructional time lost is more valuable than the performance standards. 

■ Vocational teachers need release time on a weekly or monthly basis to keep up on 
the record keeping. 

■ As teachers, we need three things: 1 . More time 2. More time 3. More time 

■ Teachers need time to do record keeping. It would not be unreasonable for each 
vocational teacher to have one period/day to do record keeping. 

■ The ADE Performance standards reporting process is cumbersome and very time 
consuming, both on the part of the course/program instructor and the local 
evaluation coordinator. It must be made less cumbersome. 



Criticism 

« The Performance Standards and Measures instrument we are currently using is 
one of the worst instruments I have had to work with. 

■ The current Performance Standards measurement are a very poor way to evaluate 
the competency attainment of individual students. This form can be made to say 
what the instructor wishes it to say! I think this is a very poor evaluation tool and 
it should be restructured. My local evaluation team members also feel that this 
system is worthless. 



DISCUSSION 

The Performance Standards Procedures survey was designed to examine LEA procedures 
used in gathering and reporting Performance Standards data. Analyses of these procedures 
allows one to draw some conclusions regarding the reliability and validity of Performance 
Standards data. 



Measuring Basic and Advanced Academic Skills 

Regarding Measure 1 (Standards 1.1 and 1.2), the results of the Performance Standards 
Procedures survey reveal that there is variation among local districts and schools regarding 
the identification and assessment of basic and advanced academic skills. Additionally, 
there is some variation among programs at the same district/school in procedures used to 
identify and assess academic skills. 

Although a majority of sites indicate using a uniform procedure to identify both basic and 
academic skills in VTE courses and programs, the procedure most often used is "teacher 
judgment." Even if all teachers are instructed to identify and assess academic skills in a 
similar way, the process is subject to personal variation and bias. There is no uniform 
process for identifying academic skills in VTE courses. 

Additionally, although a majority of sites indicate using a uniform procedure to assess both 
basic and academic skills in VTE courses and programs, the procedure most often cited is 
"teacher-developed tests." Again, even if teachers were equally competent in constructing 
good assessment instruments, the process to assess students' academic gains is largely 
subjective. 

Determinations of academic gain are made largely on the basis of comparing students' 
pretest performance with posttest performance. However, definitions of what constitutes 
"gain" vary among and between districts and programs. Most districts report a one 
percent or one-answer pre-posttest difference in scores. Such a difference is statistically 
insignificant and, most likely, practically insignificant as well. Very few districts/programs 
reported more stringent measures of gain (e.g., more than a five percent pre-posttest 
difference in scores). Where supplemental measures of "gain" or mastery are used, 
respondents most frequently report the use of teacher observations and judgment of 
student work. 

In sum, the state's database on VTE students' academic performance consists largely of 
teacher-reported perceptions of student gains on teacher-developed tests that measure 
teacher-identified skills, without shared definitions of what constitutes "gain." This, of 
course, is consistent with common practice in the schools in most subject areas, but it 
does not reflect the more objective standards that are explicit or implicit in the VTE 
Performance Standards. 

Measuring Occupational Competency 

Regarding Measure 2 (Standards 2.1 and 2.2), the results of the Performance Standards 
Procedures survey reveal that there is variation among local districts and schools regarding 
the identification and assessment of occupational competencies. Additionally, there is 
some variation among programs at the same district/school in procedures used to identify 
and assess competencies. 

Based on the survey results, over half of all respondents indicated using ADE-validated 
competency lists in order to identify course and program-related competencies. However, 
the determination of competencies to be measured from these lists is largely a matter of 
instructor discretion. 
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In determining the numbers and percentages of "course completers"— data necessary for 
reporting on the attainment of occupational competencies for Measure 2— districts vary in 
their interpretations of who is to be considered a "course completer." Only 35 percent of 
the respondents indicated using mDE's definition of a course completer, as provided in the 
Teacher's Guide (although some respondents used this definition in combination with other 
definitions). 

o 

There is variability in who constitutes a "program completer." Given the average program 
of three courses over a three-year period, and the most prevalent definition of a course 
completer as one who earns credit, a VTE program completer can be a student who earns 
credit in three related VTE courses. Coupled with the fact that a majority of proqrams (52 
percent) indicated that program completers are not required to obtain applied and 
occupational experience in the industry for which they are preparing, the degree to which 
bona fide occupational skills are attained through program completion is subject to 
question. 

Survey reporting trends suggest that the state's database on VTE students' attainment of 
occupational competencies consists largely of teacher-reported perceptions of student 
performance on teacher-determined competencies taken from ADE-validated task lists, 
where definitions of course and program completers are subject to variation across 
programs and schools. 

Measuring Continued Enrollment 

Given the ADE/VTE definition of a "course completer" as one who enrolls during the first 
grading period and remains enrolled through the end of the course, it seems reasonable to 
assume that course enrollment is similarly defined in terms of students enrolling in a 
course within the first grading period. Twenty-seven percent of the respondents used this 
definition of course enrollment. Others considered course enrollment only in terms of 
students who were in a course the entire term or who completed course requirements. 
The tendency of some districts to exclude students who might enroll during the first 
grading period and then drop the course means that districts' reporting of course 
enrollments are not equivalent. 

For Standard 3.1, the ADE Teacher's Guide directs some attention towards inservice and 
training to staff to support keeping students in school. A total of 96 percent of the survey 
respondents said such activities were of moderate to low priority. Nearly half of the 
respondents (49 percent) were dissatisfied with the emphasis (or lack thereof) placed on 
keeping students in school, and half believed their programs to be only somewhat or not 
very effective in maintaining student enrollment in school. 

Coupled with these findings, nearly half of the respondents (49 percent) do not have a 
management information system (MIS) to assist them in tracking students. Tracking and 
follow-up methods vary considerably from district to district, with at least several 
respondents indicating that student tracking is not conducted or conducted, at best, 
informally (e.g., by "word-of-mouth"). 
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Regarding Special Needs (i.e., IVEP) Students 



Results from the Performance Standards Procedures survey suggests that three out of 
every four program directors believe that special needs students are proportionately 
represented in their programs. Survey findings suggest that many VTE instructors are 
often involved (64 percent) in developing IVEPS, but use them often only 38 percent of 
the time. Additionally, only 20 percent of instructors are believed to use IVEPs in frequent 
coordination with other instructors working with IVEP students. 

One interesting finding is that nearly half of the respondents (44 percent) indicated that a 
majority of VTE program instructors do not believe that IVEP students can perform as well 
as regular students, and yet most respondents (85-88 percent) said that IVEP students are 
assessed on basic and advanced skills and occupational competency attainment in the 
same way as for regular students. 

These findings, coupled with earlier results, suggest the following scenario: Many teachers 
hold the belief that IVEP students can not perform equally as well as regular students. 
They assess teacher-identified skills/competencies using teacher-developed tests/measures 
without altering assessment methods for IVEP students. Based on research correlating 
teacher expectations with student performance, the implications for IVEP students in this 
scenario are not promising. 

Of the 1 2 percent of respondents who indicate modifying methods to the needs of special 
students, modifications ranged from watered-down curricula and lower standards of 
performance to providing additional time and assistance in mastering skills and 
competencies. 

Other Procedures Related to the Conduct of Performance Standards Evaluations 

In most cases (86 percent), the district's VTE director acts as the local evaluation 
coordinator. In the remaining cases, respondents indicated other teachers or administrators 
are charged with this task. Most respondents, too, have the ADE-required local evaluation 
teams, but not all teams have a business and industry representative or a special 
populations representative, as per ADE directives. 

Respondents indicated that most VTE faculty were provided with district or state training 
to some extent with respect to conducting the Performance Standards local evaluation, 
but over half (55 percent) said that teachers never or only occasionally were provided with 
information obtained as a result of the evaluation. 

Some 30 percent of all evaluation teams reflected in this survey either did not 
receive (or did not report receiving) training regarding their role in conducting the 
Performance Standards evaluation. Moreover, of the teams that were provided training, 
"training" consisted largely of brief lecture-type meetings. Only ten percent of respondents 
with local evaluation teams reported multiple-day or ongoing training. Nevertheless, most 
respondents (69 percent) felt that their teams understood their role in developing a local 
improvement plan and their right to submit a minority report. 
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User Evaluation of the Performance Standards System 

Overall, the user evaluation of the Performance Standards system was constructed to 
examine users' perceptions of its usefulness, accuracy and importance. Data were 
analyzed by clusters of items and items pertaining to specific Performance Standards and 
procedures. 

Clustered Items 

Items 6.1, 6.3, 6.6, and 6.8 address the importance of Performance Standards data. Items 
6.2, 6.4, 6.5, 6.7 and 6.9 refer to the ability to collect and report accurate data. Items 
6.10-6.12 examine users' opinions of the importance and reasonableness of having a local 
evaluation team. Finally, the last three items (6.13-6.16) were designed to assess user 
opinions of the overall usefulness of Performance Standards. 7 

Analyzing mean scores for clusters of items, users agree that they can accurately report 
data (mean = 3.10). Based on the overall mean of 2.78, there is borderline agreement 
that data are important (mean = 2.79). Looking at the individual items that comprise this 
cluster, however, users agree that occupational competency data are important as well as 
academic achievement data, but disagree as to the value of school enrollment data and the 
separate reporting of data for IVEP students. 

Overall, users agree with the importance and reasonableness of expectations related to 
having local evaluation teams (mean = 2.81), although this mean score is close to the 
overall mean of 2.78. 

Finally, users disagree that the Performance Standards system, as currently constructed, is 
useful (mean = 2.26). 

Items Related to Specific Standards/Procedures 

Items 6.1 and 6.2 ielate to the importance and accuracy of gathering student academic 
achievement information. Users agreed that the Performance Standards related to 
academic achievement are important (mean = 2.85) and that they can accurately 
determine whether or not students actually gain academic skills, both basic and advanced 
(mean = 2.88). 

Items 6.3 through 6.5 relate to the importance and accuracy of gathering information on 
the attainment of occupational competencies. Users agreed that this information is 
important (mean = 3.15) and that they can accurately whether course and program 
completers attain 80 percent of the competencies related to their programs (mean = 3.10 
regarding course completers; 3.05 regarding program completers). 

Items 6.6 and 6.7 examined the importance and accuracy of continued enrollment data. 
Users disagreed that the information is important (mean = 2.62), but agreed that they can 
report this information accurately (mean = 3.19). 



Items 6.13 was not included in "cluster" analyses. 
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Items 6.8 and 6.9 asked for users' opinions about the importance and accuracy of 
information related to IVEP students. Users tend to disagree that separately tracking IVEP 
students for VTE program evaluation is important (mean = 2.52). However, users felt 
confident that they are able to accurately determine the number of IVEP students in their 
programs (mean = 3.27). 

Items 6.10 through 6.12 related to the value of having a local evaluation team. As noted 
in the cluster analyses, users tend to agree that the teams play an important role in the 
district {mean = 2.95) and that it is reasonable to expect teams to have teacher, 
business/industry, and special needs populations representatives (mean = 2.88). There 
was lesser agreement on whether the district would have a team if not for the state 
requirement to do so (mean = 2.61)- 

The last four items (6.13 - 6.16) asked for users' opinions on the overall worth and 
usefulness of the Performance Standards system. Users disagreed that the collection of 
Performance Standards data is reasonable (mean = 2.64) and that the standards will be 
useful for making program improvements, once the "bugs" are worked out (mean = 2.73). 

Users also disagreed that the costs of gathering and reporting data (e.g., in terms of time 
and money) will be justified by the benefits of the system to their school or district (mean 
= 2.20). Furthermore, they would not recommend that other states use a VTE 
Performance Standards system like Arizona's (mean = 1.85). 
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PART THREE 



IMPROVING PROGRAM QUALITY 

(THE VTE MO ^ELJ 
AND PROGRAM ACCOUNTABILITY 
(PERFORMANCE STANDARDS) 



This section examines assumptions that appear to underlie Arizona's implementation of the 
VTE Model and the Performance Standards. It further discusses issues pertinent to the use 
of Performance Standards data to evalu^. *DE-funded "Model" sites. 

An assumption adopted in this report is that the VTE Model represents the state's goal for 
what a VTE Program should look like. In effect, it is the Arizona Department of Education's 
vision of a quality vocational technological education program. In parallel fashion, the 
Performance Standards system is the department's means of measuring what happens to 
VTE students as a result of VTE education. 

A premise that joins these two systems is that //sites implement a Model program, then 
their students (having experienced a higher quality program) will perform better than 
students who do not participate in a "coherent sequence of instruction." Better student 
performance should be reflected in the Performance Standards outcomes data. That is, 
"Model sites" should show more gains in students' academic achievement and attainment 
of occupational competencies. "Model sites" should manifest better rates of continued 
enrollment and higher numbers of students who continue their education and training or 
enter the workforce. It seems reasonable to assume that one should be able to assess the 
VTE Model — representing program quality— through the Performance Standards- 
representing program accountability. 

The above is a portrayal of how program quality and accountability should work in an ideal 
world. However, the Morrison Institute study of VTE Model implementation and local 
procedures used to conduct Performance Standards evaluations reveals that practice does 
not reflect an ideal world. 

The assumption that comprehensive (i.e., Levels I, II and III) VTE Model programs have 
features that significantly differentiate thern from non-comprehensive programs does not 
bear out. Although comprehensive and non-comprehensive programs do manifest some 
differences, these differences appear to fade out by Level III. And, while ADE-funded 
"Model sites" meet more of the ADE Model program characteristics than do nonfunded 
sites, funding appears to distinguish sites primarily in terms of facilities and equipment 
rather than in the instructional techniques (e.g., applied occupational experiences) that are 
more likely to have a direct effect on student performance. In sum, the distinctions 
between "Model" and "non-Model" sites are subtle, at best. To expect differences in 
student performance between "Model" and "non-Model" sites may not be realistic at this 
point in time. 
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Even if one could expect to see higher student performance at Model sites, significant 
differences in student performance could only be meaningfully detected using comparable 
data. The analysis of local procedures used to gather and report Performance Standards 
data shows that there is great variance in the ways that sites report on student academic 
achievement, the attainment of occupational competencies, and continued enrollment. 
Sites do not share common definitions of terms such as "course completer" or "program 
completer," nor are special needs students assessed equitably across districts/schools. 

Having alluded to difficulties with both the VTE Model implementation and Performance 
Standards data, the following sections elaborate on the concerns raised as a result of this 
investigation and offer some recommendations for ADE/VTE consideration. 

VTE MODEL IMPLEMENTATION 

Data from both implementation studies suggest certain areas in which ADE could direct its 
eff rts in order to refine the Model and further implement it in the schools. Suggestions 
relate to two different aspects of the Model which can be conceptualized as "Model 
problems" and "implementation problems." 

"Model problems" refer to aspects of the Model itself and of the ADE criteria used to judge 
"modelness." Several issues were identified. One issue concerns the inclusion of Level I in 
the sequence of instruction. A second issue concerns the subjectivity of ADE criterion 
statements used to define program characteristics. Finally, the relevance of certain ADE 
Program Characteristics was called into question. 

When "Model problems" are taken as a whole, they imply a need to simplify the Model. 
This report suggests that the Model can be simplified in terms of its program 
characteristics. Some characteristics can be eliminated, or combined into "assurances." 
Other characteristics need better definition. Finally, Level I itself, as well as its relationship 
to other levels, should be reexamined. These ideas are discussed in more detail in the 
paragraphs that follow. 

A recommendation to eliminate some of the Model program characteristics is prompted by 
the apparent lack of relevance and validity of certain program characteristics. That is, 
some characteristics do not adequately differentiate between types of programs (e.g., 
comprehensive and non-comprehensive); others seem unrealistic. At Level I, characteristics 
that may be unrealistic include the length of the course, the number of students served, 
leadership development at this level, and industry realism of the instructional activities, 
facilities and equipment. At all levels, characteristics that fail to significantly distinguish 
between programs include those that pertain to administrative support, teacher inservice 
and facili ation of instruction, and group work by students. 

"Shared" characteristics (i.e., those common to all three levels), especially those that do 
not meaningfully differentiate between programs, might better serve as "assurances" than 
program characteristics. "Unique" characteristics should be reduced to a core set of 
essential elements, each with indicators that specify the extent or degree of 
implementation and reduce subjective judgments. 
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There are also problems relative to terminology. By way of explanation, the ADE Program 
Characteristics Checklist was used in designing the VTE Model Components Questionnaire 
in Study 1 and formed the basis of interviews and site visits in Study 2. Questionnaire 
findings suggest that program characteristics are defined differently by personnel at 
different sites. These findings were corroborated by site evaluators. Four site evaluators 
participated in Study 2, and although researchers compared notes and developed shared 
definitions of certain program characteristics, many of the items were subject to 
interpretation. In short, the inter-rater reliability of judging program characteristics is 
subject to question. 

Questionable terminology included phrases such as "all occupations," "skill development," 
"ADE-approved" and "simulated business and industry environment." For example, schools 
using industry-defined curricula that were not produced by ADE may or may not consider 
their curriculum as "ADE-approved." Additionally, there are degrees of "industry 
simulation." In the health occupations area, there are classrooms, classrooms with hospital 
beds and equipment, and simulated doctors' offices. Clearly the latter meets the ADE 
criterion. The question concerns the presence of some equipment and variable definitions 
of "simulation." In all cases of questionable terminology, the point is that self/external 
raters are limited to an "all-or-nothing" classification. 

Finally, there are issues pertaining to the sequencing of the Model. Data reveal numerous 
implementation problems related to Level I of the Model — problems which prevented many 
sites from meeting the criteria for a fully implemented comprehensive program. The 
question is whether or not this is an "implementation problem" or a problem related to the 
conceptualization of a "coherent sequence of instruction." This report does not have 
sufficient information to answer this question, but does suggest that ADE/VTE personnel 
should reexamine how broad career exploration courses are genuinely integrated with more 
specific occupational training. The issue of whether or not a Level I course should be a 
prerequisite should be considered. The requirement for 100 percent of students being 
served is not realistic, given the organization of school sites and mobility of student 
populations. 

Additional data that point to a need to reexamine the sequencing of instruction are found 
in the LEA Performance Standards Procedures survey. The survey shows that a typical 
program consists of three courses, each one year in length. If, in fact, these programs 
represent Model programs, then students are receiving one Level I course, one Level II 
course and one Level III course. Combined with the information that most programs do not 
incorporate Model strands or applied occupational experiences in the industry for which 
students are preparing, one may question whether a three-course sequence provides 
sufficient in-depth occupational training to allow a student to enter the workforce 
prepared. 

During some site interviews, personnel raised the concern that implementing 
comprehensive Model programs forced "thinner" coverage of occupational training. This 
concern deserves attention. An implementation problem associated with this Model 
problem concerns the articulation (or lack thereof) of Level I programs with higher level 
programs. Clearly, if the Model remains with a Level I requirement, the department and 
local educational agencies need to better articulate Level I with Levels II and III. Problems 
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associated with poor articulation include a lack of policies between non-unified districts 
and their feeder schools and/or lack of administrative support within unified districts. 

In contrast with "Model problems/' "implementation problems" refer to areas in which 
program personnel appear to have difficulty meeting ADE criteria. Key implementation 
problems focus on curriculum issues (e.g., the integration of Model strands) and industry- 
realism in materials, activities, and facilities and equipment. 

At a minimum, based on the key findings of this analysis, ADE/VTE personnel should: 
■ Simplify the Model. 

V Eliminate Model program characteristics that fail to distinguish between types of 
programs. 

V Reconsider and adjust several of the Model program characteristics. Specifically, 
department personnel should develop indicators or benchmarks for each program 
characteristic. This would allow for determining the extent of implementation 
rather than the "all or nothing" judgments currently reflected by the checklist. 

V Consider the amount of instructional time available for VTE courses, given general 
graduation requirements, and whether requiring three years of broad (Level I ) to 
specific (Level III) VTE training is most appropriate and realistic within that 
framework. 

V Better articulate Level I with Levels II and III. 



Increase technical assistance and support to local educational agencies toward 
implementing the VTE Model and specifically: 

V Provide additional training to school-based VTE personnel on the six Model strands 
and on the ADE Model program characteristics. 

V Expand funding and/or technical assistance to develop appropriate multi-purpose 
environments that simulate those of business and industry. 



The current study of VTE Model implementation revealed that many Arizona school 
districts were not implementing the VTE model well at Level I, but that they were more 
successful at implementing it at Levels II and III. By addressing the areas outlined above, it 
seems likely that ADE can help Arizona school districts to better implement the VTE Model 
and to further improve their VTE programs. 
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PERFORMANCE STANDARDS EVALUATIONS 



An overall analysis of the survey results on local procedures used to conduct Performance 
Standards evaluations lends insight on issues related to the validity and reliability of the 
state's Performance Standards database. These issues are critical for ADE/VTE 
consideration, given the desire to construct an evaluation system that utilizes Performance 
Standards data. 

In measurement systems, validity concerns the extent to which a test measures what it is 
supposed to measure. Reliability concerns the consistency, dependability, or stability of 
the test score. Valid data are those that represent a meaningful measurement of a skill or 
skills; reliable data are those that can be consistently replicated. Based on the findings of 
the Morrison Institute survey, the current Performance Standards system does not 
consistently produce valid or reliable data with respect to: students' academic skills, 
students' attainment of occupational competencies, and continued enrollment. 
Additionally, the results of the survey suggest that data reported for IVEP students on the 
Performance Standards forms are not comparable across districts. 

The system— the state's database — is comprised primarily of subjective judgments that are 
based on a variety of local procedures and definitions and are not comparable. 
Furthermore, based on survey findings, training to use the system has not been adequate. 

Users appear supportive of an evaluation system that collects and reports achievement and 
occupational competency attainment data. Hov,ever, users do not judge the present 
system to be worth the costs (time and effort) and they would not recommend it to others 
seeking to increase state accountability. Several users articulated their concerns that the 
present system is both too complex and too burdensome. 

ADE/VTE personnel should consider implementing a simplified system that collects 
standardized and authentic data on students' academic and occupational skills. Without at 
least interval level data (as opposed to the currently-reported nominal and ordinal level 
perceptual data), meaningful longitudinal studies are virtually impossible to conduct. Taken 
as a whole, the findings suggest a need to redesign the database that comprises the 
Performance Standards system and implement complementary programs of technical 
assistance to assist LEAs to implement the reporting system(s). 

In proposing a redesign of the data collection system, lessons can be learned from 
research in the health field regarding surveillance systems, or systems designed to track 
the progress of a particular "health event." 8 Surveillance systems are designed to allow 
practitioners to intervene in the course of the event in order to promote better outcomes. 
This is the goal of program improvement, as well. 

Researchers have defined several components of an effective surveillance system. These 
are components that need to be in place to maximize the utility of the data for local and 
state practitioners and policymakers. An effective system — 



This discussion is adapted from the report Child and Adolescent Injury Surveillance Issues: A Feasibility Study 
by Vandegrift, J.A., Donaldson, J. & Wright, A. (1993). Tempe, AZ: Arizona State University, Morrison Institute for 
Public Policy. 
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Reflects a commitment from practitioners and policymakers. 



■ Incorporates training in its implementation and operation plan. 

■ Is cost-efficient, 

■ Uses a simple and uncomplicated format for collecting data. 

■ Anticipates change and designs data collection/analyses accordingly. 

■ Allows for longitudinal tracking, 

* Constructs a database that allows significant problems to be identified. 

* Responds to community-defined issues. 

■ Collects pertinent information (e.g., demographic and observational data) that may 
help to explain trends and/or differences in results. 

■ Bases analyses and interpretation on reliable and standardized data. 

■ Presents information in a timely fashion. 

Arizona's Performance Standards system does embody some elements from the above list. 
For example, local practitioners appear to have a philosophical commitment for a state 
data collection system, even though they may not be satisfied with the current process. 
With respect to the above list, the most serious concerns regarding the Performance 
Standards system pertain to the complexity of the reporting format, the absence of reliable 
and standardized data, and the inability to use data meaningfully for longitudinal studies. 
These concerns should be addressed. 

One promising avenue for standardizing the collection of academic achievement data 
pertains to the Arizona Student Assessment Program (ASAP). During the past year, 
ADE/VTE personnel, in concert with local vocational directors/teachers and others, began 
to explore the merger of the state's ASAP system with the VTE accountability system. 
The department should aggressively pursue this avenue of inquiry and research, as well as 
examine alternatives to documenting occupational skills attainment, other than via teacher- 
reported perceptions. Since districts and schools must employ ASAP measures anyway, 
these measures provide a viable channel for developing a credible reporting system that 
may serve to reduce burden on local districts. 
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The following are recommendations related to the Performance Standards system. 
ADE/VTE should: 

■ Redesign the Performance Standards system to enable the collection and reporting of 
valid and reliable student performance data. 

V Continue to pursue the use of ASAP data as a means to document the academic 
skills of VTE program students. 

V Explore alternative means to document the attainment of students' occupational 
competencies, so as to produce comparable data. 

V Define terms that are essential to accurate data reporting. 

V Identify existing state and local databases that provide comparable data pertinent 
to Performance Standards (e.g., the state dropout reporting system). 

■ Enhance technical assistance efforts to enable sites to collect and report more valid 
and reliable data and to use data for program improvement. 

V Encourage/provide more training on appropriate curricular and assessment 
adaptations for special needs students. 

V Provide increased technical assistance to site personnel to ensure their use of new 
and existing definitions pertinent to the collection and accurate reporting of 
Performance Standards data. 

V Enhance the training of local personnel on the use of existing state and local 
databases that provide comparable data pertinent to Performance Standards (e.g., 
the state dropout reporting system). 

V Encourage local district staff to share evaluation results with VTE staff, and assist 
local staff to meaningfully use evaluation data for program planning and 
improvement. 

The operational problems associated with both the VTE Model and the Performance 
Standards should not be construed as a condemnation of either. There are clear federal 
and state directives, as well as local support, behind the ideals for which they stand. 
Clarifying and redefining certain aspects of both the Model and the Performance 
Standards, and enhancing technical assistance to local <ites in implementing both, should 
only serve to improve program quality and accountability. 
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PROBLEMS AND PROSPECTS RELATED TO USING PERFORMANCE STANDARDS DATA 
FOR VTE MODEL PROGRAM EVALUATION 



Outcomes versus Process 

Beyond the specific difficulties related to implementing the VTE Model and the 
Performance Standards system, there are other, more conceptual, issues that pertain to 
using Performance Standards data for VTE Model Program evaluation. Although there are 
many evaluation models and frameworks, this report adopts a framework that views 
evaluation in terms of either outcomes or process. 

Outcomes evaluations are "summative" in nature. They provide information in terms of 
student performance (or other outcomes) from which one can make inferences about the 
effectiveness of an intervention (e.g., curriculum, program, strategy). In contrast, process 
evaluations are more "formative" in nature, examining how an intervention (e.g., 
curriculum, program, strategy) works on a day-to-day basis. 

Formative evaluations— that examine processes— are most useful in program improvement 
because they provide direct and specific feedback about the intervention. Summative 
evaluations— that examine outcomes — are not particularly useful in providing feedback 
about program processes. 

The VTE Model, in essence, is a process model intended to promote change in the ways 
that VTE programs operate. Performance Standards data are clearly designed to obtain 
outcome measures of student performance. The issue in question is the extent to which 
outcomes — Performance Standards — can be used to meaningfully evaluate processes — VTE 
Model programs. 

There is no question that Performance Standards data (outcomes) can be used for 
summative evaluation. However, even the most ideal Performance Standards data would 
not provide evaluation data on processes related to implementing VTE Model programs. 

The observation that the VTE Model encompasses processes is derived on the basis of 
examining ADE's program characteristics used to define "modeiness." Many of these ADE- 
defined characteristics relate to operational aspects of the programs (e.g., district and local 
support, teacher inservice). These program operations, depending upon how they are 
implemented and manifested, can contribute to improved student performance. However, 
their effective or ineffective implementation cannot be detected solely through an 
outcomes evaluation. As the VTE Model is conceptualized and operationalized, a process 
evaluation is called for in order to examine program effectiveness. This does not mean that 
an outcomes evaluation is inappropriate; rather, it means that boxh types of evaluation 
should be used. 

An outcomes evaluation can be used to meaningfully compare programs given the 
supposition that outcomes result from improved processes (e.g., research has shown that 
the instructional process of "time on task" is directly correlated with improved student 
performance; a process such as teacher inservice is not directly correlated). These 
examples demonstrate that improved processes must be directly aligned with expected 
outcomes so as to reinforce these outcomes. 
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As part of Morrison Institute's examination of the Performance Standards and VTE Model, 
analysts compared Performance Standards (outcomes) with VTE Model Program 
Characteristics (processes). This side-by-side comparison allows one to examine outcomes 
and processes at a glance. 

Table 3-1 on the following page illustrates the Performance Standards and VTE Mode! 
Program Characteristics most aligned with the standards. The table shows that only seven 
of the Model characteristics (1, 2, 4, 7, 8, 9, and 1 1) appear to directly support the 
Performance Standards outcomes. This suggests that an outcomes evaluation alone that is 
based on Performance Standards is unlikely to yield appropriate data that measures VTE 
Model Program Characteristics (processes). 
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Directions for the Future 



As part of Morrison Institute's amended 1993-94 contract with ADE/VTE, analysts 
indicated that they would provide an evaluation design — including a plan, procedures, and 
instruments— for future ADE/VTE use in evaluating the effectiveness of VTE Model 
programs. At this point in time, however, it is clear that issues related to both the VTE 
Model and the Performance Standards system are complex and require further elaboration 
and study. Both the Model and the Performance Standards are in need of revisions in order 
to use them beneficially for an outcomes evaluation. 

Furthermore, there is new federal legislation that changes the landscape of educational 
reform, including vocational technological education, and has relevance to evaluation 
design. Goals 2000, including the National Skills Standards Act of 1994, and the School- 
to-V\fork Opportunities Act of 1994, as well as sections of the reauthorization of the 
Elementary and Secondary Education Act of 1965, signify changes in the conduct of 
programming, student assessment, and program evaluation. 9 Until these changes are 
fully understood in relation to Arizona's' existing systems, specifications for evaluation are 
premature, at best. 

Morrison Institute was awarded a contract with ADE/VTE for 1994-95 to continue its 
investigation of VTE Model Implementation, Performance Standards, and evaluation issues. 
In continuing this effort, Morrison institute proposes to work collaboratively with ADE/VTE 
and local vocational personnel to create an evaluation design and system that will best 
serve local and state practitioners and policymakers. One goal of the design effort is to 
assist the state in defining/creating a valid and reliable state database aligned with federal 
legislative mandates. A second goal is to ensure that local vocational program personnel 
can use the system easily, and that local evaluation results are clearly usable and 
beneficial in program improvement efforts. 

In summary, this section suggests that Morrison Institute, in collaboration with state and 
local VTE personnel: 

■ Work toward designing an evaluation system(s) that incorporates recent federal 
and state policy reforms. 

V Examine the alignment of the Performance Standards (outcomes) and VTE 
Model Program Characteristics (processes) with respect to evaluation design. 

V Implement a process evaluation in addition to the Performance Standards 
outcomes evaluation with respect to VTE Model Program implementation. 



Wirt, John G. (1 993). Performance Assessment Systems: Implications for a National System of Skill Standards. 
Washington, DC: Training and Employment Program, Employment and Social Services Policy Studies, Center for 
Policy Research and National Governors' Association. 
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SUMMARY OF 
RE COMMENDA TIONS 



With respect to the Vocational Technological Education Model — 
■ Simplify the Model. 

V Eliminate Model program characteristics that fail to distinguish between types 
of programs. 

V Reconsider and adjust several of the Model program characteristics. 
Specifically, department personnel should develop indicators or benchmarks 
for each program characteristic. This would allow for determining the extent 
of implementation rather than the "all or nothing" judgments currently 
reflected by the checklist. 

V Consider the amount of instructional time available for VTE courses, given 
general graduation requirements, and whether requiring three years of broad 
(Level I ) to specific (Level III) VTE training is most appropriate and realistic 
within that framework. 

V Better articulate Level I with Levels II and III. 

« Increase technical assistance and support to local educational agencies toward 
implementing the VTE Model, and specifically: 

V Provide additional training to school-based VTE personnel on the six Model 
strands and on the ADE Model program characteristics. 

V Expand funding and/or technical assistance to develop appropriate multi- 
purpose environments that simulate those of business and industry. 



eric 



With respect to the Performance Standards system — 

■ Redesign the Performance Standards system to enable the collection and reporting 
of valid and reliable student performance data. 

V Continue to pursue the use of ASAP data as a means to document the 
academic skills of VTE program students. 

V Explore alternative means to document the attainment of students' 
occupational competencies, so as to produce comparable data. 

V Define terms that are essential to accurate data reporting. 
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V Identify existing state and local databases that provide comparable data 
pertinent to Performance Standards (e.g., the state dropout reporting system). 

■ Enhance technical assistance efforts to enable sites to collect and report more 
valid and reliable data and to use data for program improvement. 

V Encourage/provide more training on appropriate curricular and assessment 
adaptations for special needs students. 

V Provide increased technical assistance to site personnel to ensure their use of 
new and existing definitions pertinent to the collection and accurate reporting 
of Performance Standards data. 

V Enhance the training of local personnel on the use of existing state and local 
databases that provide comparable data pertinent to Performance Standards 
(e.g., the state dropout reporting system). 

V Encourage local district staff to share evaluation results with VTE staff, and 
assist local staff to meaningfully use evaluation data for program planning and 
improvement. 



With respect to using Performance Standards for evaluating VTE Model Programs — 

■ Work toward designing an evaluation system(s) that incorporates recent federal 
and state policy reforms. 

V Examine the alignment of the Performance Standards (outcomes) and VTE 
Model Program Characteristics (processes) with respect to evaluation design. 

V Implement a process evaluation in addition to the Performance Standards 
outcomes evaluation with respect to VTE Model Program implementation. 
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APPENDIX A 



Model-Funded Sites 



Sources of Information for Table A-1 



■ 1990-91: 

V "Vocational/Technological Education 1990-91 Model Sites Funding" (02/07/91) 
[ADE printout] 

V "Vocational/Technological Education Model Sites Fiscal Year 1991" (08/20/91) 
[ADE printout] 



■ 1991-92: 



V Snyder, L, Johnson, M., DeMuth, L & Tasker, J. (1992). Model Site Evaluation 
FY 1991-92. Tempe, AZ: Arizona State University, Vocational Education. 



■ 1992-93: 

V "1992-93 Model Sites" (01/27/93) [ADE printout] 

V ADE copies of district funding applications 

V DeMuth, L. & Tasker, J. (1993). Final Report: Arizona Vocational/Technological 
Comprehensive Program Evaluation FY 1993. Tempe, AZ: Arizona State 
University, Department of Vocational Education. 



■ 1993-94: 

V ADE copies of district funding applications 
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APPENDIX B 



Site Visit Reports and Checklists 



Type of Program 



Site District/School/Occupational Area/Program 
# 



Comprehensive- 
Model-Funded 



1 Peoria Unified/lronwood HS/Health Occupations/Nursing 
Assistant 

2 Mohave UHSD/Kingman HS/Trade and Industry/Video 
Production 

3 Buckeye UHSD/Buckeye HS/Agriculture/Business & 
Management 



Cornprehensive- 
Not Model-Funded 



4 Fountain Hills Unified/Fountain Hills Jr/Sr High 
School/Trade and Industry/Radio & TV Production 

5 Miami Unified/Miami HS/Home Economics/Child 
Development 

6 Whiteriver Unified/Alchesay HS/Business/Administrative 
Support 



Non- 
Comprehensive/ 
Model-Funded 



7 Deer Valley Unified/Deer Valley VTech/Health 
Occupations/Nursing Assistant 

8 Flagstaff Unified/Coconino HS/Trade and 
Industry/Electronics 

9 Valley UHSD/Valley Union HS/Agriculture/Business & 
Management 



Non- 
Comprehensive/ 
Not Model-Funded 



10 Yuma UHSD/Kofa HS/Trade and Industry/Drafting 

1 1 Apache Junction Unified/Apache Junction HS/Home 
Economics/Food Production 

12 Holbrook Unified/Holbrook HS/Business/Administrative 
Support 
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Site: 1R0NWQ0D HIGH SCHOOL, Peoria Unified School District 

Program: Health Occupations— Nursing Assistant 



SETTING 

The Health Occupations area at Ironwood High School was selected to represent a Model- 
funded site in an urban area. Ironwood is one of four high schools in the Peoria Unified 
District. Peoria has two Level hi programs in Health Occupations, Nursing Assistant and 
Health Assistant. This report focuses more on the Nursing Assistant program because the 
researchers interviewed an instructor from it and visited the facility for it. 

METHOD 

Information about the program was collected primarily by interviewing the Program 
Director for Health Care Technology at Ironwood High School, who is also one of two 
instructors in the Nursing Assistant program. The researchers also toured the classroom 
set aside for the Nursing Assistant program, reviewed documents provided by the program 
director, and talked briefly with one student and the other teacher from the program. The 
interview with the program director was audiotaped, and a transcript of the audiotape was 
used in writing this report. 

ORGANIZATION 

Level I 

Level I is at the seventh and eighth-grade levels, not at the high school level. It is a year- 
long course that is available to all students in grades 7 and 8, but it is not required. The 
Level I course is not considered prerequisite to Level II and Level III courses. The Level I 
course is not articulated with the Level II and III courses, apparently because Level I is in 
the elementary schools and Levels II and III are in the high schools. 

Level II 

The name of the Level II course for the Health Occupations is Medical Science. This year- 
long course meets for an hour a day and is open to students in grades 10-12. The course 
covers 40 to 50 health occupations, as well as topics such as first aid, pregnancy, 
sexually transmitted diseases, etc. The Level II Medical Science course is offered at all four 
high schools in the district. It is a prerequisite for entry into the district's Level III Health 
Occupations courses. 

Level III 

Peoria has two Level III courses in the Health Occupations area, Nursing Assistant and 
Health Assistant. Enrollment is limited to 50 students from the district in each course. 
Admission to both programs is competitive and requires a written essay, an interview, 
testing, and letters of reference from teachers. The 50 students in each program are 
divided into two classes of 25 each. Students are selected from the four high schools for 
both programs. The two Nursing Assistant classes are held at Ironwood High School and 
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the two Health Assistant classes meet at Peoria High School. Students from other high 
schools provide their own transportation to these classes. 



The Level III Nursing Assistant and Health Assistant courses meet for two hours daily 
throughout the school year. Students in these courses must have completed the Level II 
Medical Science course, but a Level I course is optional for them. The Level III Nursing 
Assistant course provides skill development for employment as a nursing assistant and 
leads to employment in this occupation and to advanced training in nursing-related 
occupations. According to the program director, many of the Nursing Assistant students 
are employed as nursing assistants or in related positions during the summer and after 
graduation, and 87 percent of the graduates have gone on to a community college or 
university. Nearby Glendale Community College offers an Associates Degree in Nursing 
and the state universities offer Bachelor's and advanced degrees in the nursing field. 

Local and District Support 

The program director reported that her principal is "100 percent" supportive of the health 
occupations program, and that the other principals are also very supportive. She indicated 
that the Peoria superintendent is very vocational oriented. She was less confident about 
the school board's support for vocational programs, primarily because of limited contact 
with them. She had recently made a presentation to the school board, and she thought 
that they reacted quite positively to it. 

Advisory Council 

The program has an advisory board that consists mainly of administrators from area 
hospitals. The board meets twice a year, near the beginning and end of the school year. 
The advisory board provides advice on the skills and knowledge needed by nursing 
assistant students and on practicum and employment opportunities. The program director 
feels that the board has an excellent commitment to the program. 

CURRICULUM 

The curriculum is competency based in the sense that there is a list of competencies and 
skills for both the Level II Medical Science course and the Level III Nursing Assistant 
course. Performance on the competencies at both levels is assessed primarily with paper- 
and-pencil tests, although some are laboratory skills monitored by the instructor. Student 
progress through the curriculum is tracked through the competencies list and through tests 
that are entered into the gradebook or onto the computer, 

The Level III Nursing Assistant Curriculum was written by the program director, who 
reported that the skills and competencies are updated yearly. The list of competencies is a 
comprehensive one that is not referenced to ADE academic and occupational 
competencies. The program director said that she "looked at" the six model strands when 
she developed the curriculum, but that her basic reference was a textbook that is 
successful in helping students pass the Arizona certification exam for Nursing Assistant. 
She reported that the Nursing Assistant students have a 100 percent pass rate on this 
exam. 
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The curriculum provides for student leadership development through a required 24-week- 
long, two-day-a-week practicum, placement of students in summer and post-graduation 
jobs, and through student participation in Vocational Industrial Clubs of America/Health 
Occupations Education. The VICA/HOE group meets for a half hour during class time on 
Fridays. They do volunteer work, run a health clinic during lunch hour every Wednesday, 
hold fund raisers and conduct other nursing-related activities. 

INSTRUCTIONAL DELIVERY 

The program director for Health Care Technology teaches a Level II Medical Science course 
and a Level III Nursing Assistant course. The Level II course provides students with a 
variety of experiences related to the Health Occupations area. Students in the Level II 
course learn about different health occupations, learn medical terminology and body 
systems, study various illnesses and diseases, learn lab safety procedures, and have 
frequent speakers from the health field. 

The Level III course provides experiences more specifically associated with the occupation 
of nursing assistant. The teacher manages a variety of iearning experiences that include 
acquisition and demonstration of numerous skills related to client care, effective 
communication and team membership, and performance of nursing assistant duties in an 
applied setting. These activities include industry-realistic situations and materials to the 
extent that safety and a limited budget permit. They also include realistic occupational 
experiences provided by the 24-week , two-days-a-week practicum for each student. 

IVEP Students 

The admissions criteria for the Nursing Assistant program essentially precludes the 
admission of IVEP students. The program director reported that she feels that the program 
is a demanding one, and she indicated that it may be unrealistic to admit students who 
may have difficulty going on to postsecondary education. She indicated that some special 
needs students, such as hearing impaired and limited English proficiency students, are 
admitted to the program and that appropriate adjustments are made to accommodate their 
needs. 

FACILITIES AND EQUIPMENT 

The facility for the Nursing assistant program is one large classroom with two small 
connected rooms, one containing computer equipment for use with the program and one 
serving as an office for the two Nursing Assistant instructors. The classroom contains a 
variety of equipment of the type found in a medical clinic, medical teaching setting or a 
hospital. The classroom is reasonably well equipped for its purpose, but the environment 
does not simulate one that would be found in a well-funded medical clinic, laboratory or 
hospital. The equipment and materials clearly are based on the curriculum, even though 
they are not of a standard that would be common in the health industry in this country. 

The program director reported that ADE funds enabled her to create and develop the 
Nursing Assistant program. She said she believes "we would have never implemented this 
program without that (ADE) funding." She indicated that she'd had only about $1000 a 
year from the district for the past four years, but that the district is revising its funding 



scheme and she expects to get more funding in the future. She also noted that hospitals 
had given them much of their equipment. 



TEACHER QUALIFICATIONS 

There are two teachers at Ironwood High School who teach the Level II and Level III Health 
Occupations courses and a total of six Level II and Level III Health Occupations instructors 
in the Peoria district. All six are certified in Vocational Education with a Health 
Occupations endorsement. Both Level III Nursing Assistant instructors are also registered 
nursed who remain current in their field by working an "on call" basis for two or three 
days a month as nurses in area hospitals. The district Level II and Level III Medical Science 
teachers meet approximately once a month and the Level III teachers meet more 
frequently. They have three professional days a year to attend workshops and update their 
skills and knowledge. 

COMMENTS 

The Nursing Assistant program in the Health Occupations area at Peoria appears to be a 
strong program. This Level III program is closely linked to the prerequisite Level II course in 
Medical Science. Level I courses in Peoria are offered in the elementary schools and do not 
have a defined relationship to the higher-level co.urses. The Level III Health Occupations 
courses in Peoria were developed from personal experience and knowledge of the 
instructors and from textbooks and instructional materials familiar to them. The ADE Model 
Strands had little or no apparent influence on their development. 

The Nursing Assistant program appears to be well supported by the district administration 
from the perspective of encouragement and enthusiasm, but the funding for equipment 
and supplies is meager. ADE support has been crucial to the development and success of 
the program. The instructors have also done a good job of obtaining equipment from area 
hospitals. 

The program director cited several strengths of the program in response to question from 
the researchers. A primary strength was the dedication of the instructors. "They all love 
what they do," she said. "We really, really enjoy teaching these classes and we have a fun 
time teaching them. ..We like teenagers, we like helping people." She indicated that she 
does not know what has been prompted as a model (by ADE), but that it is important to 
have "the equipment that matches the competencies in the curriculum." She also noted 
that she is very pleased with the availability of training sites in the nearby area and the 
"terrific support out in this northwest community." 
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IRON WOOD HIGH SCHOOL, Peoria Unified: Health Occupations— Nursing Assistant 



Program Area and Characteristics 



4. 



LEVEL 
I II III 



yes 



yes 



yes 



I. ORGANIZATION 

1. Instruction provides: 

• an exploration/skill development for all occupations. 

• skill development for a cluster of related occupations. 

• skill development for specific occupation(s), 

2. Students served: 

• 100% in the targeted grade level (7-9). 

• In the targeted grade levels (9-10) have previously completed a Level I 
experience. 

• Students have the opportunity to participate in more than one cluster, if 
desired. 

• In the targeted grade levels (1 1-12) have completed Level I and II 
experiences. 

3. The experience is a minimum of: 

• one year in length or equivalent to 225 min/week. 

• 360 hours in length. 
The experience provides a coherent sequence of instruction: 

required for all vocational technological prog r ams. 
each cluster leads to at least one specific Level III experience, 
leads to employment and advanced training. 

The experience has the support of the local school/district administration. 



no 



yes 



no 



yes yes 



yes 



no 



no 



5. 
6. 



An active advisory council assists in designing and implementing this 
experience. 



II. CURRICULUM 

7. Curriculum is competency-based with each student's attainment of each 
competency monitored. 

8. Curriculum is designed to deliver all academic and occupational 
competencies (approved by ADE) in: 

• the draft Level I Framework. 

• at least one cluster. 

• specific occupation(s). 



yes 

ns yes yes 
ns yes yes 

yes yes yes 

ns 



yes 



yes 
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Program Area and Characteristics 



LEVEL 
I II III 



ns no 



yes 



13. Students work in groups or teams frequently. 

14. Instructional activities include industry-realistic situations and materials; and 
are evaluated according to industry-related standards. 

15. The instructional delivery includes occupational experiences which can 
include shadowing and cooperative education. 

IV. FACILITIES AND EQUIPMENT 

16. The instructional facility provides a multi-purpose environment with 
appropriate technology and support systems. 

17. The environment simulates one that would be found in business and 
industry. 

18. The equipment and materials utilized in the experience are determined on 
the basis of the curriculum and industry standards, 

V. TEACHER QUALIFICATIONS 

19. The teacher(s) are vocationally certified in: 

• vocational areas or if a team of teachers instruct the Level I experience, at 
least one will be vocationally certified. 

• appropriate vocational areas related to the specific cluster content. 

• the specific vocational area. 



ns 



yes 



yes 



no 



ns yes yes 



II. CURRICULUM - continued 

9. Curriculum includes all six of the Model "Strands. K 

10. Curriculum provides for student leadership development, which can include 
the use of vocational student organizations. 

ill. INSTRUCTIONAL DELIVERY 

1 1 . The instruction provides students with experiences in: 

• a variety of occupational areas. 

• occupational areas associated with the particular cluster(s). 

• specific occupation(s), 

12. The teacher serves as a facilitator of instruction, managing a variety of 
learning experiences for students. 



yes 



ns yes yes 

ns yes yes 

ns yes yes 

ns ns yes 



ns yes yes-. 



ns no no 



ns yes yes 



I 

I 
I 

I 

I 

1 
1 

11 

n 
i 



yes 



20. The teacher(s) have recent inservice to update and upgrade their teaching 
techniques and content knowledge. 



yes 



yes 



ERLC 



B~8 



Q9 



Site: KINGMAN HIGH SCHOOL, Mohave Union High School District 
Program: Trade & Industry— Video Production 



SETTING 

The Video Production program at Kingman High School in the Mohave Union High School 
District was selected to represent a Model-funded "comprehensive" program. Kingman 
High School is located in a rural district in northwestern Arizona, Mohave County. 

METHOD 

Data collection consisted of interviews with the Vocational Director, the Video Production 
teacher, and the Mohave Community College ITV Technician, who is also a member of the 
District Advisory Committee. A tour of the facility was conducted and written and video 
documentation was gathered. Documents provided for review included the following: 

1. Course Description Booklet/Registration Information 

2. Course Competencies 

3. Vocational/technological Curriculum Framework: Level I 

4. Tech-Prep Articulation Agreement 

5. Video Production Proposed Tech-Prep Program 

6. Lesson Plans: Video Production/Media English and Video Production Orientation 

7. Newspaper Article: TV Yearbook Production 

8. Kingman High School Video Yearbook 

This report results from an analysis of staff interviews, the documentation, and the facility 
tour. 

ORGANIZATION 

Levels Nil 

Level I programs are offered through a course titled "Tech Lab-Odyssey 2000." The course 
is offered at the ninth grade level and is not a required course for all students. It is one 
year in length and explores careers in such areas as life management, administrative 
services and technology. The curriculum utilizes the ADE materials and covers the six 
strands. Particular effort has been expended by the district to develop the strands for use 
in the Level I program. 

Competencies are defined and measured in the program, which utilizes ADE competency 
lists. The course is taught by a vocationally certified teacher who is also a mathematics 
teacher. A teaching aide also works with the program, for which the school was given a 
special grant to teach mathematics, technology and the strands. No vocational student 
organizations (VSOs) are available to students at this level. 

Level II, Video Production/English, involves students in a year long program. Students may 
unroll in the program without having taken a required exploratory course (Level I). Students 
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are given 1/2 credit in English and 1/2 credit in electiva. The class is team taught by a 
vocationally certified teacher and an English instructor. It is offered at grades 10-12. 

The curriculum utilizes the ADE strands and a modified ADE curriculum which includes 
teacher-selected units. ADE competencies and teacher-developed competencies are 
utilized. The class creates and produces the Kingman High School announcements 
broadcast through the school television studio. A junior HERO program is available to 
students and generally does not include students from the Video Production/English class. 

Level III, Video Yearbook (Video Production II), is a stand-alone class that is offered at 
grades ten through twelve. A prerequisite for the year-long program includes at least one 
of the following: Jr. High Video Class, Video Communication, Photo I or II class. 
Exceptions are made to the prerequisite requirement based on teacher approval. The 
program utilizes the six strands and ADE curriculum, as available. Practicum experiences 
are not offered but off-campuS^"shoots" are heavily utilized. Student activities are videoed 
and a video yearbook is produced to sell on campus. (A copy of the Video Yearbook was 
reviewed as part of the documentation.) The course is team taught by a vocationally 
certified teacher and an art teacher. 

Local and District Support 

Persons interviewed agreed that there was moderately high support (7-8 on a scale of 10) 
for the programs at the district level and a slightly lesser level of support with campus 
administration. One administrator commented that "If I have to sacrifice something 
(because of funding and/or scheduling), it will be in Level I electives." Staff comments 
were positive except to note that scheduling is a problem and they would like more time 
for class (block time). 

Advisory Council 

Two advisory groups assist the Video Production Program. One of these is an internal 
committee consisting of school administration, English, Math, Video, and Technology 
teachers and the vocational director for the district. 

An external advisory committee consisting of a representative from the community 
college, several business representatives and two employers assist the Video Production 
Program. This committee is "100% committed, great". They meet two or three times a 
year to review the curriculum, equipment and progress of the program. In particular, the 
community college representative and several other committee members assist in the hioh 
school program periodically on a volunteer basis. 

CURRICULUM 

Kingman's Level III programs include Agricultural Business and Management, 
Accounting/Computing Occupations, Administrative Support, General Marketing, Child 
Care, Food Production Management, Institutional Home Management, Auto Mechanics, 
Drafting, Welding, Building Trades, and Communications/Media. Each of these programs 
has at least one course at Level III, but the courses are not considered necessarily as a 
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"comprehensive program" by district administration. Level II programs include Business 
Management Technology and Industrial Technology. 

The vocational programs utilize the ADE strands and curriculum materials and teacher- 
developed materials. Within the Communications/Media program, high priority is given to 
the job skills needed in video production. 

Vocational student organizations available to students include VICA, STRIVE, FBLA and 
FFA. VICA, in particular, serves video production students. Participation in VSOs is 
voluntary. 

Student competencies are described in the course and unit objectives and are defined and 
measured through teacher assessment. Students arp encouraged to obtain employment 
after graduation and/ or to continue in the nearby community college program. A 60-62% 
follow-up return rate one year after graduation indicated that a large number of students 
were continuing in the video production field. 

Student progression through the vocational program(s) is tracked through the Counseling 
Office and by the individual teacher(s) as well. Teachers maintain competency attainment 
records. 

INSTRUCTIONAL DELIVERY 

Interviews with instructors, an advisory committee member and the vocational director 
confirmed that project-oriented, hands-on instruction is emphasized. Course objectives, 
program descriptions and units of instruction further confirmed that there is a strong 
emphasis on acquiring competencies, particularly in English, Math and Video Production. 
Applied skills as well as occupational skills have priority in the Level II and III programs. 

IVEP Students 

The program is available to special needs students, including LEP/academically 
disadvantaged students. Competencies are not defined and measured in the same way for 
these students. Competency lists are not included at present in the IVEP, but will be 
included during the FY 94-95 school year. 

TEACHER QUALIFICATIONS 

All teachers working with the Level I and II program are vocationally certified, with the 
exception of two teachers in the business area. One industrial Technology teacher has a 
provisional certificate. The Video Production teacher is vocationally certified. As noted 
earlier, the video program is team taught with a certified art and/or English teacher. The 
Vocational Director indicated that "certification assistance is needed by ADE" in the 
technology area. 

The district provides support for teachers to attend the August vocational conference, 
summer institutes, model site meetings and tech consortium meetings. A one-half day 
inservice on outcome-based education was provided by the district. In addition, the video 
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production instructor was given financial assistance to addend the CAMDE Trade Show, 
the National Association of Broadcasters and SBE meetings. 

FACILITIES AND EQUIPMENT 

Kingman High School moved into a newly constructed facility in the Fall, 1993. Thus, the 
facility is particularly modern and well-equipped. The Video Production 
classroom/laboratory is excellent and the district indicated that ADE funding has allowed 
development of this program, as well as others. The vocational director stated that "State 
funding has motivated teachers to take the model seriously." 

The most current equipment is available to students, including roving television recorders, 
video tape recorders and film editors. A studio/sound stage is utilized by students in 
production activities and other off-campus activities are available. 

COMMENTS 

The program at Kingman High School nearly meets all of the criteria for a fully 
implemented model comprehensive program. The most notable exception is that there is 
no enforced requirement that students have met the prerequisite standard for entry into 
Level II and III programs. There are ongoing district efforts to develop a definite sequence 
of courses, to integrate the ADE strands into the curriculum and to develop more discrete 
lists of competencies and indicators for the various program levels. 

The district has developed an articulation agreement with Mohave Community College and 
is working to finalize the tech prep programs in Computer Technology and Video 
Production. The programs have strong board and community support, excellent facilities, 
an enthusiastic staff, and high student interest. Having adequate instructional time is a 
minor problem. 
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KINGMAN HIGH SCHOOL, Mohave UHSD: Trade & industry -Video Production 



Program Area and Characteristics 



LEVEL 
i II III 



yes 



yes 



yes 



no 



I. ORGANIZATION 

1. Instruction provides: 

• an exploration/skill development for all occupations. 

• skill development for a cluster of related occupations. 

• skill development for specific occupation(s). 

2. Students served: 

• 100% in the targeted grade level (7-9). 

• In the targeted grade levels (9-10) have previously completed a Level I 
experience. 

• Students have the opportunity to participate in more than one cluster, if 
desired. 

• In the targeted grade levels (1 1-12) have completed Level I and II 
experiences. 

3. The experience is a minimum of: 

• one year in length or equivalent to 225 min/week. 

• 360 hours in length. 

4. The experience provides a coherent sequence of instruction: 

• required for all vocational technological programs. 

• each cluster leads to at least one specific Level III experience. 

• leads to employment arid advanced training. 

5. The experience has the support of the local school/district administration. 



no 



yes 



no 



yes yes 



yes 



ns 



ns 



yes 



yes 
yes 



An active advisory council assists in designing and implementing this 
experience. 



II. CURRICULUM 

7. Curriculum is competency-based with each student's attainment of each 
competency monitored. 

8. Curriculum is designed to deliver all academic and occupational 
competencies (approved by ADE) in: 

• the draft Level I Framework. 

• at least one cluster. 

• specific occupation(s). 



yes yes 



yes yes yes 



yes 



yes 



yes 
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Program Area and Characteristics 



LEVEL 
i I! Ill 



H. CURRICULUM - continued 

9. Curriculum includes ail six of the Mode! "Strands." yes yes yes 

10. Curriculum provides for student leadership development, which can include 

the use of vocational student organizations. ns yes yes 

III. INSTRUCTIONAL DELIVERY 

1 1. The instruction provides students with experiences in: 

• a variety of occupational areas. yes 

• occupational areas associated with the particular clusters). yes 

• specific occupation(s). yes 

1 2. The teacher serves as a facilitator of instruction, managing a variety of 

learning experiences for students. yes yes ves 

13. Students work in groups or teams frequently. yes yes yes 

14. Instructional activities include industry-realistic situations and materials, and 

are evaluated according to industry-related standards. yes yes yes 

15. The instructional delivery includes occupational experiences which can 

include shadowing and cooperative education. ns ns ns 

IV. FACILITIES AND EQUIPMENT 

16. The instructional facility provides a multi-purpose environment with 

appropriate technology and support systems. ns yes yes 

17. The environment simulates one that would be found in business and 

industry. ns yes yes 

18. The equipment and materials utilized in the experience are determined on 

the basis of the curriculum and industry standards. ns yes yes 

V. TEACHER QUALIFICATIONS 

19. The teacher(s) are vocationally certified in: 

• vocational areas or if a team of teachers instruct the Level I experience, at 

least one will be vocationally certified. yes 

• appropriate vocational areas related to the specific cluster content. yes 

• the specific vocational area. yes 

20. The teacher(s) have recent inservice to update and upgrade their teaching 

techniques and content knowledge. ns yes yes 
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SUe: BUCKEYE UNION HIGH SCHOOL, buckeye Union High School District 
Program: Agriculture Education— Agricultural Business & Management 



SETTING 

The Agricultural Business and Management program at Buckeye High School was selected 
to represent a Model-funded "comprehensive" vocational-technological program. Buckeye 
is a union high school district in a rural environment in Maricopa County. Two teachers are 
responsible for the delivery of the program. 

METHOD 

Data collection consisted primarily of staff interviews. The high school principal acts as the 
vocational coordinator of the programs and was met with briefly. The teacher who has 
chief responsibility for the instruction and coordination of the agriculture programs at 
Buckeye was the key interviewee. Dcing the site visit, a comprehensive tour of the 
facilities was given by one eleventh-grade and one twelfth-grade student. Each of them 
were involved in the program and had aspirations of continuing their education in a parallel 
career field. The interview with the instructor was audiotaped; the student tour of the 
facility was videotaped. In addition, documents were provided for review and included: 

1. "Biotechnology and Ag Business Management Program" (program of study) 

This document outlines the course offerings in this program. It includes units of 
instruction for each course, student competencies for each course, and classroom 
safety regulations. The document also includes a curriculum guide which illustrates 
the objectives of each course and how each course aligns with the corresponding 
Arizona Essential Skills. Activities and evaluation criteria are also delineated. 

2. Student Competencies portfolio 

This document portrays the means by which the agriculture teachers at Buckeye 
monitor and assess the competencies developed by each of the students enrolled in 
the Ag/Business management program. Competencies include those related to 
leadership development, animal science, plant science, and specific skill areas 
addressing agricultural construction. 

This report results from an analysis of staff and student interviews, the facility tour, and 
the program documents. 

ORGANIZATION 

Levels /-Iff 

Level I consists of a course entitled "AG and the FFA" (Future Farmers of America). The 
course is exploratory in nature and is an elective open to ail students, although ninth grade 
students are encouraged to enroll. Students explore animal and plant science, along with 
recordkeeping and other business-related skills, using computer networks and file servers 
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purchased with ADE funds. Two "independent labs" are interconnected by the networks. 
These two labs, one in the business department and one in the agriculture department, 
jointly serve students in the program. 

Staff note that the first two years of student enrollment in agriculture business "are really 
exploratory." A Lab 2000 program is also offered that was described as a "great 
exploratory tool." The linkage between the Lab 2000 program and the Ag/Business 
management program was not specified. However, Lab 2000 was not considered an 
essential element of the Level I program. 

Level li consists of a course entitled "Plant & Animal Science." Level I is required for the 
program, if the student is to be a four year completer. Students completing the Level !l 
course earn a science credit toward meeting the high school's graduation requirements. 
Asked how Level II is differentiated from Level I, staff note: "[There isn't] too much of a 
difference. [Students] gain more knowledge in Level II. It's building on what they've 
learned in their exploratory courses." 

The courses offered at Level III include Biotechnology, Livestock Production Management, 
Landscape Design and installation, Crop Production Management and Ag Business 
Management, 

Level I and level II courses are considered preparatory but not necessarily prerequisites for 
enrollment in advanced level courses. Level III courses have a strong emphasis on science 
and the department is currently discussing with the district administration the possibility of 
granting a lab science credit for successful completion of these courses. 

There is considerable collaboration between the agriculture faculty and the science faculty 
in relation to the courses, most specifically the Biotechnology course. Comparing science 
and agriculture laboratory facilities, the agriculture teacher stated: "We have better quality 
equipment due to the [ADE] grant." In addition, in developing and designing the biotech 
lab, Buckeye staff consulted with a technician at the University of Arizona. According to 
the teacher, "We modeled after U of A." 

Local and District Support 

The overall district support of the program was rated as a seven on a scale from one to 
ten. There was some concern expressed about policies regarding class enrollment. At 
present, classes are combined when class sizes are too low to meet district minimums. 
Because of having classes combined due to limited enrollment, there is some concern that 
"the program [is] not as comprehensive" as staff would like. Although the district was felt 
to have been supportive in the past, staff expressed a desire "to see more leniency in 
[class size] numbers," particularly since there are plans to develop new programs. 

With a growing district enrollment and an expanding program, staff expressed hope that 
the district would consider hiring another agriculture teacher. Because of limited district 
resources, however, there was doubt that this would happen. In other areas, such as 
facilities and equipment, program personnel felt that there was adequate funding: "[The 
district is] real cooperative; we have an adequate budget to fund our current program." 
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Advisory Council 

FFA alumni and employees and owners of local business and industry act as Buckeye's 
advisory group for the Agricultural Business and Management program. The group's 
membership includes approximately 30 individuals who assist in curriculum review and 
project evaluation. According to program personnel: "They'll let you know if you're not 
doin' things right. 1 ' 

Reportedly, the group has been very active in the development of the biotechnology 
program. The group meets monthly. Asked about the level of support from the advisory 
committee, the instructor stated: Td rate 99% of them a 10 [with 10 being high]. They 
are real supportive. If I need something done and I can't physically do it myself, they'll 
help me find a way to get it done." The group a*so helps generate funds for scholarships 
for graduating agriculture students. 

CURRICULUM 

Buckeye's curriculum guide and course outlines illustrate the scope of the curriculum for 
each of the courses within the AG/Business program. These materials demonstrate the 
linkages of course objectives with Arizona Essential Skills. When asked about the 
integration the VTE Model's six curriculum strands, program personnel expressed 
unfamiliarity with these strands. 

Leadership development is addressed in both the classroom curriculum and vocational 
student organizations (VSOs). While FFA is the primary VSO, students and instructors also 
have worked with Future Business Leaders of America (FBLA) and the school's Total 
Quality Management (TQM) program. These program linkages are intended to broaden 
agriculture students' awareness of the world of business and enable them to experience 
the connections between agriculture and business. VSOs were described as "a very 
important part [of the program]; it's a great motivational tool." 

Each student's competencies are determined by the teachers through observation and 
project completion. According to the instructor, "All these tasks tie in to some kind of 
project. Each individual student will have a project. Each of the skills has a terminal 
objective which is to create a project utilizing these skills." 

All competencies are measured according to projects completed at each of the three 
levels. While the teacher assigns projects in Levels I and II, student-initiated projects are 
commonplace and expected at the advanced Level III. 

Students are individually monitored and evaluated by the instructors. Students are tracked 
and identified as completers when they have successfully accomplished each of the 
competencies outlined in the portfolio. 

In conjunction with the University of Arizona, the school has developed a computerized 
Program Completer Report that enables high school staff to follow the progress of their 
graduates. Progress is monitored one and five years following graduation. 
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INSTRUCTIONAL DELIVERY 



Through brief observation of classrooms, an interview with the teacher, and conversations 
with students, it appeared that the majority of classroom instruction in the Level II and III 
courses focused on providing hands-on, project based experience for students. The 
instruction also emphasized the relevance of academic subject content, i.e., mathematics 
and science, with applications in agriculture and business. 

IVEP Students 

When asked how instruction accommodates the special needs of IVEP students, the 
teacher responded, "I don't necessarily single them out. In the skill development area, I 
expect them to attain as well as they can. In fact, I've had some IVEP students that. ..do 
better than the other kids. One is now our shop manager. He's learned a lot of agricultural 
skills hands-on, which is a great vehicle for any student." 

Instructional aides are also provided by the district to aid with Limited English Proficient 
students. The same competency expectations are held of all students, regardless of 
classification. 

TEACHER QUALIFICATIONS 

Both teachers are permanently certified in agriculture education. The teacher interviewed is 
pursuing his master's degree in agricultural education at the University of Arizona. He is 
also active in developing Ag contests at the state level. He has participated in TQM 
training and has been a guest speaker. The other instructor is very active with the county 
and state fairs, serving on the board of directors. The district has offered inservice on 
topics such as alternative teaching strategies. However, time committed to in-service was 
eliminated during the 1993-94 academic year to address issues related to the Arizona 
Student Assessment Program, i.e., curriculum alignment with state Essential Skills. Both 
agriculture teachers regularly participate in the state vocational training workshops. 

COMMENTS 

Overall, the teacher described the Ag/Business program at Buckeye as a "complete 
program." Its strengths are in the quality of the equipment and laboratories, its positive 
relations with local business and industry, and its appropriateness as a career alternative 
for students. To improve the program, a physical expansion of the facility is needed, i.e., 
an off-site field for livestock. Also, the school is presently expanding its landscaping 
program, which could require another teacher to manage the program. The biggest 
limitation, according to the teacher, is time. 

With the exception of the lack of a "required" Level I course and the absence of the 
curriculum strands, the program at Buckeye meets most of the ADE criteria for a 
comprehensive program. Once again, as at other sites, the levels are defined as 
preparatory experiences for each other. 

The teacher seemed to have great respect for the members of the advisory group. 
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BUCKEYE HIGH SCHOOL, Buckeye UHSD: Agriculture -Business & Management 



Program Area and Characteristics 



LEVEL 
II III 



yes 



yes 



yes 



no 



I. ORGANIZATION 

1. Instruction provides: 

• an exploration/skill development for all occupations. 

• skill development for a cluster of related occupations. 

• skill development for specific occupation(s). 

2. Students served: 

• 100% in the targeted grade level (7-9). 

• In the targeted grade levels (9-10) have previously completed a Level I 
experience. 

• Students have the opportunity to participate in more than one cluster, if 
desired. 

• In the targeted grade levels (11-12) have completed Level I and II 
experiences. 

3. The experience is a minimum of: 

• one year in length or equivalent to 225 min/week. 

• 360 hours in length. 

4. The experience provides a coherent sequence of instruction: 

• required for all vocational technological programs. 

• each cluster leads to at least one specific Level III experience. 

• leads to employment and advanced training. 

5. The experience has the support of the local school/district administration. 



no 



yes 



yes 



yes 



yes 



yes 



An active advisory council assists in designing and implementing this 
experience. 



II. CURRICULUM 

7. Curriculum is competency-based with each student's attainment of each 
competency monitored. 

8. Curriculum is designed to deliver all academic and occupational 
competencies (approved by ADE) in: 

• the draft Level I Framework. 

• at least one cluster. 

• specific occupation(s). 



no 

yes 

yes 

yes yes yes 
yes yes yes 



yes yes yes 



yes 



yes 



yes 
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Program Area and Characteristics LEVEL 
1 II III 

II. CURRICULUM - continued 

9. Curriculum includes all six of the Model "Strands." no no no 

10. Curriculum provides for student leadership development, which can include 

the use of vocational student organizations. yes yes yes 

III. INSTRUCTIONAL DELIVERY 

11. The instruction provides students with experiences in: 

• a variety of occupational areas. yes 

• occupational areas associated with the particular cluster(s). yes 

9 specific occupation(s). ye:, 

12. The teacher serves as a facilitator of instruction, managing a variety of 

learning experiences for students. yes yes yes 

13. Students work in groups or teams frequently. yes ns yes 

14. Instructional activities include industry-realistic situations and materials, and 

are evaluated according to industry-related standards. ns no yes 

15. The instructional delivery includes occupational experiences which can 

include shadowing and cooperative education. ns ns yes 

IV. FACILITIES AND EQUIPMENT 

16. The instructional facility provides a multi-purpose environment with 

appropriate technology and support systems. yes yes yes 

17. The environment simulates one that would be found in business and 

industry. ns ns yes 

18. The equipment and materials utilized in the experience are determined on 

the basis of the curriculum and industry standards. yes yes yes 

V. TEACHER QUALIFICATIONS 

19. The teacher(s) are vocationally certified in: 

• vocational areas or if a team of teachers instruct the Level I experience, at 

least one will be vocationally certified. yes 

• appropriate vocational areas related to the specific cluster content. yes 

• the specific vocational area. yes 

20. The teacher(s) have recent inservice to update and upgrade their teaching 

techniques and content knowledge. yes yes yes 
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Site: FOUNTAIN HILLS JUNIOR-SENIOR HIGH SCHOOL, Fountain Hills Unified School 
District 

Program: Trade and Industry— Radio/TV Production 



SETTING 

The Radio/TV Production program at Fountain Hills Junior-Senior High School was selected 
as representative of a self-reported comprehensive Trade and Industry program that has 
never received additional ADE "Model site" funding. A new school, and the only secondary 
school in the Fountain Hills Unified School District, this site is located in a suburban 
community several miles to the northeast of Scottsdale. 

METHODS 

The data were collected through individual interviews with the high school principal and 
the radio/TV production teacher, and a tour of the instructional facility. Two course syllabi, 
one for the introductory course and one for the advanced course in video production, were 
also provided. This site was selected as a late replacement for another site. As a result, 
the site visit was scheduled on short notice for the teachers' last day at school before 
summer break. This report was derived from an analysis of the interviews, the tour of 
facilities and the course syllabi. 

ORGANIZATION 

Level I 

According to the school principal, the Level I class, Introduction to Video Production, is a 
one-semester elective course open to students in grades 9-12. "It's an introduction to the 
equipment, to the terminology. The students get some experience going out and filming 
things. It's just to get their feet wet." Intro, to Video Production is a prerequisite for 
Advanced Video Production. When asked about how Level I differentiates from Levels II 
and III, the principal responded: "Level I is strictly introductory. If students want to get 
more involved, they'll sign up for the Advanced Video Class." 

The introductory video course does not address many of the ADE Level I criteria. It is an 
elective course, targeting 9th through 12th grade students, is only a semester in length 
and does not follow the ADE draft Level I curriculum framework. 

The school also offers a LAB 2000 course which is considered by staff to be the Level I 
component referred to on our questionnaire. 

Level II and Level III 

The Advanced Video Production is a year-long course which the principal defines as the 
Level II and Level III course. Students may enroll in the course for more than one year. In 
other words, the first year a student enrolls in Advanced Video Production, the course is 
considered Level II; the second year of enrollment is considered Level III. According to the 
principal, "It's the same class. It's an Advanced Video Production class. We're not blessed 
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with the teacher or the time to create four or five sections of video production, so the 
teacher will have some kids in the Advanced class for two or three years." 

Both second and third year students are enrolled in Advanced Video Production. In 
actuality, there is no Level II program in radio/TV production at this site. The advanced 
course more closely parallels Level III criteria. There is no presentation of occupational 
clusters in the course. 

Local and District Level Support 

The principal's support is demonstrated through his commitment of school time to daily 
and monthly broadcasts and of funds to support professional development. The principal 
c scribed the district support as follows: "I think in principle the support is high, but in 
carry through, it's very low. It's one thing to say 'yes, we think the video productions 
class is real great, but unfortunately you don't have any money'.... The financial support is 
very weak, but— in theory— they support the idea." 

Advisory Council 

The school has a newly formed (April 1994) site-based council which is made up of 
teachers, students, administrators, and parents. There are also some community groups 
that help with fund raising for special vocational education projects, but there is no 
advisory council, per se, for vocational education programs. 

CURRICULUM 

When asked if he knew how the curriculum covered the six strands, the principal 
answered "No I don't." The instructor's response to questions about the use of ADE 
curriculum further indicated that neither district nor state curriculum guided the video 
production program at this site. VTE curriculum strands are not utilized. 

The instructor interviewed for this report was not the individual who was responsible for 
completion of the VTE Performance Standards report. He stated that "for the most part 
I've been left out of the Vocational Education Department." In regard to evaluating student 
skill development, the teacher has developed his own set of competencies according to the 
particular project on which students are working. He referred to the course syllabi as 
giving "a general outline of those things." 

Much of the curriculum, especially in the advanced level course, is project-based. Level II 
and Level III are represented at this site by one course through which the instructor 
individualizes instruction based on the student's level of competence. The teacher actively 
collaborates with other instructors in core academic areas, and students frequently 
produce projects in the video class to meet requirements in other courses. 

The Video Club at Fountain Hills is responsible for the daily news broadcast and a quarterly 
special broadcast. Other projects undertaKen by the Video Club include a videotaped 
campus tour, a junior American History project, and a fashion show produced with the Life 
Management department. All students in the Advanced Video Production class are 
considered members of the Video Club. 



B-22 



INSTRUCTIONAL DELIVERY 



The instructional strategies used at the advanced level emphasize projects of different 
types. The students produce a daily news broadcast that is shown in each of the school's 
classrooms. They also produce monthly special programs, orientation programs for 
entering sixth grade students, and school information programs used by local realtors. The 
principal described this hands-on approach to learning as one of the program's assets: "We 
have a venue for the students to exhibit their productions on a daily basis. It's not just a 
class. ..this is a real hands-on learning experience. ...There is an avenue for them to be able 
to exhibit their goods." 

The occupational exploration is limited to fields directly related to video production, e.g., 
editing, dubbing, etc. The program does not include radio production because of the 
limited equipment and lack of a studio facility. The teacher individualizes instruction for 
students according to their level of proficiency. Students regularly work on group projects, 
although some individual projects are also pursued. Due to equipment and facility 
limitations and the prohibitive cost of upgrading, the program is far from an "industry- 
realistic situation." 

IVEP Students 

There are no IVEP students enrolled in this program. 
FACILITIES AND EQUIPMENT 

Concerns expressed by the principal and the teacher about facilities and equipment were 
validated by a tour of the facility. One classroom was used for the video production 
proc/am. Equipment was limited to a small number of VCRs (10), some low-quality audio- 
dubbing tape recorders, and one TV. Other equipment included things such as tripods and 
cabinets for storage of tapes. Some computerized graphic art software and a computer 
terminal were used for graphics production and title-making. Although the teacher had 
been very creative in his use of the facility and the principal was very supportive of the 
program, the equipment and facility restrictions severely limited this program's possibility 
of approaching industry standards. The principal reported that, "We're using what 
amounts to the standard home VHS-type camera as opposed to the VHS-S format. ..We're 
not at that level. And probably, unless some funding comes in, I doubt that we'll ever get 
to that kind of equipment in this facility." 

TEACHER QUALIFICATIONS 

The teacher who teaches the video production courses is certified in English but had 
taught a video production course in a Pinal County school for one year prior to coming to 
Fountain Hills. Neither the principal nor the instructor were aware of any certification 
specific to radio/TV production. Both noted, however, that they had not received state 
funding for their program because the teacher was not appropriately certified in vocational 
education. 

The district supports professional development of its vocational/technological teachers. 
The video production teacher attended a seminar offered by SONY which cost the district 
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around $1000. The principal "wanted him to pick up some ideas from people who are in 
the business.... It's hard to find workshops in video production put on by the state or even 
private concerns here locally. ...I'd like to see the state do more in some of these more 
newly developing areas." 

The school principal also identified district inservice training as a source of professional 
development. Inservice during 1993-94 was substantially dedicated to assessment, with 
particular emphasis on the value of portfolio and project assessment. 1 994-95 inservice 
programs will include a component addressing interdisciplinary instruction. 

COMMENTS 

The program at its current level of development represents a traditional occupational 
preparation program, incorporating twc levels of preparation — introductory and advanced. 
Many factors seem to influence the program's lack of movement toward a comprehensive 
program. These include: lack of communication with the Vocational Education program and 
ADE, limited funding, and limited development time because of the newness of the school 
and program. 

The principal described the greatest strength of the program as the teacher, who "gets the 
kids excited about the program." Nevertheless, growing enrollment has required the 
current instructor to teach English full-time in the 1994-95 school year. The district 
planned to recruit for a an additional part-time teacher for the 1994-95 Video Production 
classes. 
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FOUNTAIN HILLS JR/SR HS, Fountain Hills Unified: Trade & Industry -Radio & TV Production 



Program Area and Characteristics 



LEVEL 
I II lil 



no 



no 



no ■ 



ns 



no 



no 



yes 



yes 



no 



I. ORGANIZATION 

1. Instruction provides: 

• an exploration/skill development for all occupations. 

• skill development for a cluster of related occupations. 

• skill development for specific occupation(s). 

2. Students served: 

• 100% in the targeted grade level (7-9). 

• In the targeted grade levels (9-10) have previously completed a Level I 
experience. 

• Students have the opportunity to participate in more than one cluster, if 
desired. 

• In the targeted grade levels (11-12) have completed Level I and II 
experiences. 

3. The experience is a minimum of: 

• one year in length or equivalent to 225 min/week. 

• 360 hours in length. 

4. The experience provides a coherent sequence of instruction: 

9 required for al! vocational technological programs. 

• each cluster leads to at least one specific Level III experience. 

• leads to employment and advanced training. 

5. The experience has the support of the local school/district administration. 

6. An active advisory council assists in designing and implementing this 
experience. 

II. CURRICULUM 

7. Curriculum is competency-based with each student's attainment of each 
competency monitored. 

8. Curriculum is designed to deliver all academic and occupational 
competencies (approved by ADE) in: 

• the draft Level I Framework. 

• at least one cluster. 

• specific occupation(s). 



yes 



no 

ns 

ns yes 



no no no 



yes yes yes 



no 



no 



no 



I 
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Program Area and Characteristics LEVEL 
I II 111 

II. CURRICULUM - continued 

9. Curriculum includes all six of the Model "Strands." no no no 

10. Curriculum provides for student leadership development, which can include 

the use of vocational student organizations. ns no no 

III. INSTRUCTIONAL DELIVERY 

1 1. The instruction provides students with experiences in: 

• a variety of occupational areas. no 

• occupational areas associated with the particular cluster(s). no 

• specific occupation(s). yes 

12. The teacher serves as a facilitator of instruction, managing a variety of 

learning experiences for students. yes yes yes 

13. Students work in groups or teams frequently. yes yes ye.s 

14. Instructional activities include industry-realistic situations and materials, and 

are evaluated according to industry-related standards. no no no 

15. The instructional delivery includes occupational experiences which can 

include shadowing and cooperative education. no no no 

IV. FACILITIES AND EQUIPMENT 

16. The instructional facility provides a multi-purpose environment with 

appropriate technology and support systems. no yes yes 

17. The environment simulates one that would be found in business and 

industry. no no no 

18. The equipment and materials utilized in the experience are determined on 

the basis of the curriculum and industry standards. no no no 

V. TEACHER QUALIFICATIONS 

19. The teacher(s) are vocationally certified in: 

• vocational areas or if a team of teachers instruct the Level I experience, at 

least one will be vocationally certified. 9 no 

• appropriate vocational areas related to the specific cluster content. no 

• the specific vocational area. no 

20. The teacher(s) have recent inservice to update and upgrade their teaching 

techniques and content knowledge. yes yes yes 
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Site: MIAMI HIGH SCHOOL, Miami Unified School District 
Program: Home Economics— Child Development 



SETTING 

Miami High School was selected as a self-described "comprehensive" program that has 
never received ADE Model site funding. The site is located in Gila County in a rural area 
having somb tourist industry and copper mining as its primary income source. 

METHOD 

Data collection consisted of meeting with the high school principal, who is also the 
vocational coordinator. Additional meetings were held with the vocational staff, 
particularly the Home Economics teacher. A brief tour of the facility was also conducted. 

Documentation reviewed included the following: 

1 . Vocational Education Advisory Committee Evaluation 
(Life Management Committee) 

2. Articulation Meeting minutes (Level I program) 

3. Articulation Agreement with Eastern Arizona College 

4. Vocational Advisory Board member list 

5. Automotive Technology Level I-III graphic 
(Model for other programs) 

6. Miami High School Class Schedule 

Copies of this documentation are available for review. This report summarizes 
conversations with the respective persons and review of the documentation provided. 

ORGANIZATION 

Levels Nil 

Miami Unified District offers a Level I program for students in grades seven and eight. It is 
a career exploration program which is required for all students. The course is one semester 
in length and explores life management, building trades, keyboarding, and graphic arts 
occupational areas. There are no vocational student organizations. The course is taught by 
two teachers who are certified in home economics and woods. The Level I program is new 
for the district in FY 93-94. 

Level II programs are offered in industrial technology and administrative support areas. 
Commercial Art, Life Management, Welding, and Building Trades are available. The courses 
are one semester in length and are offered for one full year. They utilize ADE-defined 
curriculum strands and ADE curriculum materials as they are available. Other teacher- 
prepared materials are also used. Family Living units will be added during 1994-95. 
Vocational student organizations are available, including VICA and FBLA. 
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Level lit programs are available in Administrative Support, Building Trades, Auto 
Mechanics, Commercial Art, Welding, Child Development, Accounting, and General 
Marketing. The Advanced Child Development program was offered for the first time during 
FY 1993-94. Level III courses utilize ADE and teacher prepared competencies. The 
Principal has indicated that they still "need more curriculum change to provide competitive 
career skills." 

Local and District Support 

On a scale of one to ten, with ten being high, overall district support for the programs was 
rated at the 8-9 level with the exception of one program — Administrative Support, which 
was rated at the 5-6 level. While the district uses Perkins funds, it has very limited district 
funds that are used to support the programs, ADE has notified the district that they must 
improve in this effort. 

Advisory Council 

An overall district Vocational Advisory Committee is operational and meets on a monthly 
basis. It has six community, three community college, and several district ex-officio 
members. Among their duties, this committee has assisted the district to conduct ADE 
Performance Standards reviews. 

CURRICULUM 

As noted, Miami High School offers training programs in several occupational areas. The 
programs utilize some ADE materials in the Level II programs and will utilize ADE 
curriculum and strands materials in the Level III programs beginning in the 1994-95 school 
year. No practicum experiences were offered during 1993-94, but will b£ available the 
following year. Level II programs offer vocational student organization (VSO) leadership 
opportunities through VICA and FBLA. 

The district also participates in the Tech Prep coordination program with Eastern Arizona 
College to articulate curriculum. 

ADE Competency Lists and other teacher-developed competencies are used in the 
programs. Students are individually monitored by teachers. Competencies are defined and 
measured through performance and cognitive testing. 

Follow-up of program completers after high school graduation is conducted. Most 
graduates obtain employment in the mines, food services and government services areas. 
Of the students who continue their education, most go to a community college (Mesa 
Community College or Eastern Arizona College — Gila Pueblo Campus) and do not receive a 
bachelor's degree. 
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INSTRUCTIONAL DELIVERY 

The principal and vocational instructors confirm that the primary method of instructional 
delivery is through classroom projects and discussion. Hands-on opportunities are 
available. In the Child Deveiopment program, there is an excellent preschool center. The 
Life Management 1 and 2 program emphasizes non wage-earning activities. 

IVEP Students 

Permanent files for all students are maintained with the counselors. Teachers must notify 
counselors of accommodations made on the IVEPs for special needs students. Modified 
competencies are used, as needed. Adult aides are utilized to assist severe and profoundly 
disabled persons. 

FACILITIES AND EQUIPMENT 

The facilities and equipment at Miami High School are in a state of flux. The automotives 
area is well equipped; graphics will get additional equipment next as will the business 
areas. With the exception of the preschool center, the Home Economics facility is in need 
of major modifications to meet industry standards. Equipment in the area, though, is quite 
good, including two new IBMs for classroom use. 

TEACHER QUALIFICATIONS 

All teachers in the Level II and III programs have appropriate vocational certification, and 
all teachers regularly participate in upgrading their skilis through professional development. 
Professional development activities are made available through participation in the 
Vocational Summer Conference. Both the automotive and life management instructors 
have attended curriculum workshops. Additional inservice has been provided on classroom 
management and discipline practices. 

COMMENTS 

There is consensus among staff and the principal that the programs at Miami High School 
have good enrollments for electives. The district is in the process o'. upgrading several 
program areas in facilities and equipment and in curriculum and instruction. Funding for 
staff and equipment continues to be a problem. Ther* are six program areas in the 
vocational area and they "are strong and supportive of each other." In addition, the 
programs generally enjoy strong Board and community support. The district has been 
moving toward a strong Tech Prep curriculum and is working cooperatively with the 
nearby community college to articulate curriculum efforts. 

While the high school is making progress toward realizing a fully implemented "model" 
program, the Child Development program in the Home Economics area does not, to date, 
satisfy all ADE criteria for a model comprehensive program. 
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MIAMI HIGH SCHOOL, Miami Unified: Home Economics— Child Development 



Program Area and Characteristics 



LEVEL 
1 II 111 



yes 



yes 



yes 



I. ORGANIZATION 

1. Instruction provides: 

• an exploration'skill development for ail occupations. 

• skill development for a cluster of related occupations. 

• skill development for specific occupation(s). 

2. Students served: 

• 100% in the targeted grade level (7-9). 

• In the targeted grade levels (9-10) have previously completed a Level I 
experience. 

• Students have the opportunity to participate in more than one cluster, if 
desired. 

• In the targeted grade levels (11-12) have completed Level I and II 
experiences. , • 

3. The experience is a minimum of: 

• one year in length or equivalent to 225 min/we^k. 

• 360 houfs in length. 

4. The experience provides a coherent sequence of instruction: 

• required for all vocational technological programs. 

• each cluster leads to at least one specific Level ill experience. 

• leads to employment and advanced training. 

5. The experience has the support of the local school/district administration. 



yes 



yes 



no 



no yes 



yes 



6. An active advisory council assists in designing and implementing this 
experience. 

II. CURRICULUM 

7. Curriculum is competency-based with each student's attainment of each 
competency monitored. 

8. Curriculum is designed to deliver all academic and occupational 
competencies (approved by ADE) in: 

• the draft Level I Framework, 

• at least one cluster. 

• specific occupation(s). 



no 

yes 

yes 

yes yes yes 
no yes yes 



ns yes yes 



ns 



yes 



yes 
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Program Area and Characteristics LEVEL 
I II ill 

II. CURRICULUM - continued 

9. Curriculum includes all six of the Model "Strands." ns no no 

10. Curriculum provides for student leadership development, which can include 

the use of vocational student organizations. no yes yes 

III. INSTRUCTIONAL DELIVERY 

1 1 . The instruction provides students with experiences in: 

• a variety of occupational areas. yes 

• occupational areas associated with the particular cluster(s). yes 

• specific occupation(s). yes 

1 2. The teacher serves as a facilitator of instruction, managing a variety of 

learning experiences for students. ns yes yes 

13. Students work in groups or teams frequently. ns yes yes 

14. Instructional activities include industry-realistic situations and materials, and 

are evaluated according to industry-related standards. ns ns no 

15. The instructional delivery includes occupational experiences which can 

include shadowing and cooperative education. ns ns yes 

IV. FACILITIES AND EQUIPMENT 

16. The instructional facility provides a multi-purpose environment with 

appropriate technology and support systems. no no no 

17. The environment simulates one that would be found in business and 

industry. ns no no 

18. The equipment and materials utilized in the experience are determined on 

the basis of the curriculum and industry standards. ns yes yes 

V. TEACHER QUALIFICATIONS 

1 9. The teacher(s) are vocationally certified in: 

• vocational areas or if a team of teachers instruct the Level I experience, at 

least one will be vocationally certified. yes 

• appropriate vocational areas related to the specific cluster content. yes 

• the specific vocational area. yes 

20. The teacher(s) have recent inservice to update and upgrade their teaching 

techniques and content knowledge. yes yes yes 
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Site: ALCHESAY HIGH SCHOOL, Whiteriver Unified School District 
Program: Business Education — Administrative Support 



SETTING 

Alchesay High School in the Whiteriver Unified District is located in east central Arizona in 
Navajo County on the Whiteriver Apache Reservation. The school is in an isolated, rural 
area with few businesses. However, because the community of Whiteriver is the tribal 
capital, there are both tribal and federal government employment opportunities. The high 
school was selected to represent a self-reported "comprehensive" program that has never 
received ADE "Model site" funding. Two teachers are responsible for the Business 
Education— Administrative Support program. 

METHODS 

Data collection consisted of interviews with program and district personnel, including two 
business instructors, four other vocational teachers, and the vocational coordinator. The 
site e\ aluator also had a brief meeting with the principal and another meeting with seven 
students from a cross-section of vocational programs. Documentation for review included: 

1 . Business Procedures/Technology Course Outline 

2. ADE Project Proposal 

3. Student Enrollment Handbook 

4. Alchesay High School Vocational Program Video 

This report summarizes information garnered from the site visit and interviews. 

ORGANIZATION 

Level I 

Level I consists of three courses: Introduction to Technology, Life Skills, and Special Needs 
Vocational Exploration. The Level I program utilizes the Lab 2000, which will be moved to 
the Junior High School program effective September, 1 994. The program has emphasized 
manufacturing, construction, transportation, and communication. All incoming freshmen 
are required to take this half-credit course in addition to the Life Skills half-credit course. 

Life Skills focuses on entry level employment skills. TV/media will be added to the high 
school curriculum in 1995-96 through the English Department. Because of the move of the 
Lab 2000 to the middle school, a 12-station drafting program will be initiated in the high 
school. Keyboarding will be offered at the seventh grade. 

The Special Needs Vocational Exploration course allows special needs students the 
opportunity to complete the Level I course and mainstream when they have attained 80 
percent of the competencies for the course. 

Level I programs are one semester in length. 
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Level II 



The Level II course clusters include keyboarding, information processing, Shorthand I, 
bookkeeping and Typing I. These courses target competencies that are prerequisites to the 
Level 111 program in Administrative Support. Other Level II programs available include Life 
Management, Automotive, Drafting/Design Technology, Carpentry, and Health Careers. All 
Level II programs are one year in length. 

Level lit 

The Comprehensive Administrative Support Program, Level III, emphasizes word 
processing, spreadsheets, graphics, database management, and telecommunications. 
Prerequisite course work (Level II) is required for enrollment in the Administrative Support 
Program. Other Level III courses (Typing II and Shorthand II) are articulated with Northland 
Pioneer College programs. A wr.tten articulation agreement does not exist, but the 
crosswalk of competencies is in process. Level III programs are one year in length. 

Local and District Support 

Overall, district support for the programs was rated as 8-9 on a scale of 10, with 10 being 
high. The district has a new superintendent as of 1 993-94. As a result, staff are more 
optimistic about the level of administrative support. There is no written Board policy 
statement regarding vocational education, although there was a recent Board resolution in 
support of the new Global Office Systems Beyond 2000 proposal for the Administrative 
Support program. 

Students indicated that they feel they have good opportunity to enroll in vocational 
programs and they are generally satisfied with the programs. 

Advisory Council 

All programs have been reviewed by the district advisory committee. Most recently the 
Vocational Drafting proposal was developed for 1994-95 and reviewed by the committee. 
All programs are also reviewed by an in-school steering committee before becoming target 
programs. 

The district's advisory committee has representation from Indian Health Services, the 
Bureau of Indian Affairs, Apache Tribal Child Care Services, and the Tribal Apprenticeship 
and Education Offices. The committee meets every other month to review curriculum. 
They have also participated in the Performance Standards Evaluation. 

CURRICULUM 

Several changes will occur in the high school curriculum as a result of Lab 2000 moving to 
the middle school. Drafting Technology will be enhanced. An exploratory health program, 
Commercial Foods, and Apprenticeship will be permanently added in 1994-95. 
Entrepreneurship, Business Economics and Cooperative Education will be added on a 
rotating basis to the Level III program. Business English, General Office Practice, and 
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Communications are being added to the Level II program and will use a team 
teaching/cross-curriculum approach. 

In particular, the Administrative Support program utilizes ADE strands and curriculum 
materials. Other programs, including Drafting, are still developing these materials. The 
Building Trades program offers co-credit with Northland Pioneer College. 

Leadership activities for students in Levels II and III are available through two vocational 
student organizations: STRIVE and FBLA. VICA will be operational in 1994-95. 

Competencies are defined and measured in all programs through teacher tracking. In the 
Administrative Support program, students work on a contract basis. The contract 
incorporates both ADE competencies and teacher-directed activities utilizing the six 
strands. 

A district study has revealed that the Administrative Support program utilizes 
approximately 325% of the competencies with another 19 percent in place but needing 
upgrading. They have estimated that 47 percent of the competencies are non-existent and 
are working to add these to the curriculum in FY 95. Six learning/interest stations have 
been proposed that would be devoted to the strands. These would be added to the 
Administrative Support program. These stations, combined with the course additions 
described above, will provide 100 percent of the competencies and the six strands. 

The Vocational Director keeps the data base for student tracking purposes. Follow-up 
efforts have shown that nearly 75 percent of those responding are in the military, in 
college and/or working. 

INSTRUCTIONAL DELIVERY 

A simulated model office atmosphere is utilized in the Administrative Support program. 
The equipment is particularly good and represents industry standards. New local area 
network (LAN), computer hardware, and office furnishings have been proposed to further 
enhance the program. Project-based activities and student performance contracts are used 
in the program. Local advisory council members serve as guest speakers. Simulated job 
interviews, with video replay, are also utilized. 

The district has proposed networking all three business classrooms next year and utilizing 
a modem connect system in which students may communicate with other schools in 
Arizona by use of electronic mail. Proposals to update the facility include adding two 
simulated office classrooms and one multi-media learning classroom with 20 computer 
workstations. 

IVEP Students 

4// students are required to develop a four-year plan on entrance into Alchesay High 
School. The Vocational Coordinator indicated that there is not enough follow-through by 
counseling staff when scheduling, but that the problem is being corrected gradually. 
As IVEP students attain 80 percent of the competencies in the Level I program, they a r e 
transitioned to the Level II and/or III programs as appropriate. Teachers identify modified 
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competency goals for each student and measure their performance accordingly. Targeted 
state competencies are utilized, particularly for IVEP students. 

FACILITIES AND EQUIPMENT 

Major facilities/equipment plans affecting the Level III Administrative Support program 
have been described above. In general, the facilities at Alchesay High School are in need of 
preventative maintenance. Several areas also pose housekeeping problems. The new 
administration has indicated an interest in generally upgrading the facility. Shop and 
automotives areas are particularly crowded, while the business area has ample-sized 
rooms. Some of the curriculum changes described earlier will also necessitate upgrading 
several classrooms. 

TEACHER QUALIFICATIONS 

All teachers in the Level II and III programs are vocationally certificated. There is some 
question as to whether or not the middle school teacher taking over the Lab 2000 will be 
vocationally certified. 

The Vocational Coordinator is also the Special Education Transition Coordinator. The 
Administrative Support teacher is vocationally certified, as is the Coordinator. 

The Industrial Arts teacher has participated in the Lab Technician summer programs. Each 
year several vocational teachers have participated in the Vocational Conference. Outside 
resources (i.e., Deer Valley and Northland Pioneer College) have been used to provide 
training/inservice. A consultant was also hired to work with the Administrative Support 
program planning activities. Other inservice provided by the district has included how to 
work with IVEPs. Several teachers have completed or will complete by next year their ESL 
certification. 

COMMENTS 

Alchesay High School is one of the state's pioneers in using a "four period day" format, 
which extends classroom instruction time to 90 minutes per class each semester. Students 
generally like the longer period because it gives them "time to really get something done." 
Students are able to attain eight credits per year within this system. 

Staff believe that there is a need to add more career clusters and define articulation 
arrangements. The Vocational Coordinator stated that it is "very difficult to keep students 
on track and to provide course sequences because of scheduling problems." 

The district has many positive plans to further expand the Level I, II, and III programs. 
These plans are more extensive for some programs than for others. The advisory 
committee, and staff in general, seem supportive of the expansion efforts. The 
Administrative Support program instructor was particularly enthused about the network 
and learning center plans for the school/program. 

While the programs are not as well articulated as might be desirable, developing a coherent 
sequence of instruction is being addressed and corrective actions have been identified. 
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ALCHESAY HIGH SCHOOL, Whiteriver Unified: Business Education -Administrative Support 



Program Area and Characteristics 



LEVEL 
I U III 



no 



yes 



yes 



yes 



I. ORGANIZATION 

1. Instruction provides: 

• an exploration/skill development for all occupations. 

• skill development for a cluster of related occupations. 

• skill development for specific occupation{s). 

2. Students served: 

• 100% in the targeted grade level (7-9). 

• In the targeted grade levels (9-10) have previously completed a Level I 
experience. 

• Students have the opportunity to participate in more than one cluster, if 
desired. 

• In the targeted grade levels (11-12) have completed Level I and II 
experiences. 

3. The experience is a minimum of: 

• one year in length or equivalent tc 225 min/week. 

• 360 hours in length. 

4. The experience provides a coherent sequence of instruction: 

• required for all vocational technological programs. 

• each cluster leads to at least one specific Level III experience. 

• leads to employment and advanced training. 

5. The experience has the support of the local school/district administration. 



yes 



ns 



no 



no yes 



yes 



6. An active advisory council assists in designing and implementing this 
experience. 

II. CURRICULUM 

7. Curriculum is competency-based with each student's attainment of each 
competency monitored. 

8. Curriculum is designed to deliver all academic and occupational 
competencies (approved by ADE) in: 

• the draft Level I Framework. 

• at least one cluster. 

• specific occupation(s). 



yes 

yes 

yes 

yes yes yes 
ns yes yes 

ns yes yes 



yes 



yes 



yes 
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Program Area and Characteristics LEVEL 
! 11 HI 

II. CURRICULUM - continued 

9. Curriculum includes all six of the Model r Strands." ns no no 

10. Curriculum provides for student leadership development, which can include 

the use of vocational student organizations. ns yes yes 

HI. INSTRUCTIONAL DELIVERY 

1 1. The instruction provides students with experiences in: 

• a variety of occupational areas. yes 

• occupational areas associated with the paiticular cluster(s). yes 

• specific occupation(s). yes 

1 2. The teacher serves as a facilitator of instruction, managing a variety of 

learning experiences for students. yes yes yes 

1 3. Students work in groups or teams frequently. yes yes yes 

14. Instructional activities include industry-realistic situations and materials, and 

are evaluated . according to industry-related standards. yes ns yes 

15. The instructional delivery includes occupational experiences which can 

include shadowing and cooperative education. ns ns ns 

IV. FACILITIES AND EQUIPMENT 

1 6. The instructional facility provides a multi-purpose environment with 

appropriate technology and support systems. yes yes yes 

1 7. The environment simulates one that would be found in business and 

industry. yes yes yes 

18. The equipment and materials utilized in the experience are determined on 

the basis of the curriculum and industry standards. yes yes yes 

V. TEACHER QUALIFICATIONS 

19. The teacher(s) are vocationally certified in: 

• vocational areas or if a team of teachers instruct the Level I experience, at 

least one will be vocationally certified. ns 

• appropriate vocational areas related to the specific cluster content. yes 

• the specific vocational area. yes 

20. The teacher(s) have recent inservice to update and upgrade their teaching 

techniques and content knowledge. yes yes yes 
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Site: 
Program: 



DEER VALLEY VTECH CENTER, Deer Valley Unified School District 
Health Occupations— Nursing Assistant 



SETTING 



I 
I 
I 
I 
I 
1 
I 
I 
I 
I 
I 
I 



Deer Valley VTech Center is one of three schools in the Deer Valley Unified District that 
serve students in grades 9-12. The school was selected to represent a Model-funded site 
in an urban area. Deer Valley's Nursing Assistant program in the Health Occupations area 
was selected for study. Although the district has received state funds to develop a 
comprehensive model (Levels l-lll) in this area, the school indicated on Morrison Institute's 
"VTE Model Components Questionnaire" that they did not offer a "comprehensive" 
program because they did not consider either their Level I or Level II program to meet 
Model criteria. 

METHOD 

Information about the program was collected primarily by interviewing the program 
coordinator, who is also an instructors in the Nursing Assistant program. The researcher 
also toured the school campus and Health Occupations facility, and reviewed documents 
provided by the program coordinator. The researcher also had the opportunity to speak 
informally with several students and other teachers from the program. The interview with 
the program coordinator was audiotaped and transcribed. This transcript, notes, and 
program materials were used in writing this report. 

ORGANIZATION 

Level I 

The district has three middle schools serving students in grades seven and eight, A Level I 
program is in place at each middle school, and is required for all Deer Valley students 
except those enrolled in band. Students must participate in "Lab 2000" and/or "Life 
Skills." Classes are offered in nine-week blocks. Reportedly, most students take an 
average of 18 weeks of instruction in life skills and nine of the technologv skills. The Lab 
2000 experience logically "feeds into" all technology areas at Level II and II!. Life Skills is 
the Level I experience leading to advanced studies in Home Economics, Health 
Occupations, and Business Education. 

Level II 

The Level II course for Health Occupations — Medical Arts 1-2 — is offered to students in 
grades ten through twelve at both of the district's comprehensive high schools. Students 
are not required to have taken an exploratory Level I course, but staff report that most 
have completed such an experience. 

The year-long course meets for one period each day. The course's main emphasis is on 
anatomy and physiology in health and disease, nutrition, medical terminology, and career 
exploration. Medical Arts falls into the Level II "Applied Biological Systems" occupational 
cluster defined by ADE, and participating students earn science credit for completion. The 
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Level I! Medical Arts is not a prerequisite for entry into the district's Level III Health 
Occupations courses, but is highly recommended. 

Level III 

The Deer Valley VTech Center offers Health Occupations training in nursing and a number 
of allied health occupations. Programs on the VTech campus target students in grades 
eleven and twelve. The Nurse Assisting program meets for two periods each day for two 
semesters. The program is approved by the Arizona Nursing Consortium and Arizona Board 
of Nursing as a Certified Nurse Assistant program. The course offers training in basic 
nursing care skills, health care problems, the disease process, prevention and treatment. 
After the first nine weeks of instruction, students participate in a practicum at a local 
hospital twice a week for 27 weeks and provide direct patient care under the supervision 
of a registered nurse. Specific skills include emergency care, physical examinations, wound 
care, sterile techniques, personal hygiene, and a number of other treatments. 

Local and District Support 

The program coordinator reported that her principal is highly supportive of the health 
occupations program and vocational programs in general. Furthermore, she notes that "In 
reality, there's not a whole lot of people in the entire district that aren't supportive of this 
program." In fact, the Deer Valley Unified School District has a formal, written "Statement 
of Philosophy for Vocational Education" that provides evidence of the district's 
commitment. Moreover, plans are now underway to construct a new high school that will 
include a multi-million dollar "technology plaza" intended to provide "one-stop shopping" 
for information technology, industrial technology, life skills management, etc. 

The only questionable level of support had to do with the district's school board. 
According to staff, support "can depend on any particular point in time" and the program 
has experienced "varying degrees of support" over the years. 

Advisory Council 

The program has an active advisory board that consists of about 20 public and private 
sector health care providers. The committee meets twice annually to review curriculum, 
materials, and resources. They provide advice on types of equipment to purchase. In 
addition, they come on campus to "actually do teaching segments with kids." They help 
identify and coordinate site placements for clinical rotations. The program coordinator feels 
that the Advisory Board provides outstanding support. 

CURRICULUM 

All courses in the Health Occupations sequence are competency-based. The Level I classes 
reportedly incorporate ADE curriculum and use ADE competency lists. Participating 
students in Level I are assessed using portfolios. The Level II Medical Arts course 
integrates ADE and commercial (e.g., CORD) curriculum materials in instruction. It is also 
corripetency-based, and students are assessed using performance testing and competency 
check lists. Level III course competencies are clearly defined and carefully recorded for 
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each student. Competencies are prescribed based on the requirements of an examination 
administered through the State Board of Nursing. Course completers are eligible to take 
this examination. Also, successful graduates may enter a Licensed Practical Nurse (LPN) 
program through an articulation agreement with Gateway Community College. 

Student records for Levels II and III are kept on computer, allowing staff to track and 
monitor student performance. According to the program coordinator, "There'd be no way 
of tracking Level I, other than the fact that the students complete their seventh and eighth 
grade experience with Deer Valley schools," 

Regarding ADE's six curriculum strands, the program coordinator notes that for Level I 
"there are a couple of components that need to be strengthened" and that planning is 
underway to do so. At Levels II and III, these strands are "interwoven" but do not drive 
the curriculum. At Levels II and III, student leadership development is integrated into the 
curriculum through participation in VICA (Vocational Industrial Clubs of America). VSOs 
are not currently active at the middle scnool level, although efforts are underway to 
establish them. 



INSTRUCTIONAL DELIVERY 



Following ADE criteria, Level I is a broad-based career exploration course, while Medical 
Arts (Level II) prepares students for careers in nursing and allied health occupations. Level 
III provides specific occupational training. All instructors manage a variety of learning 
experiences for students. This is apparent looking at the facilities which combine 
classroom and lab areas. Based on conversations with staff and students, students often 
work in groups and teams and have access to "industry-realistic" situations and materials. 
Practicum experiences and clinical rotations, including six weeks of working in a Deer 
Valley school nurse's office (e.g., assisting school nurses conduct screening examinations 
— vision, hearing, height, weight), ensure hands-on occupational experiences. 



IVEP Students 



In the Level II and III programs, records indicate less than one percent of all students who 
begin the program drop out and 88 percent are known to obtain entry-level positions or 
pursue postsecondary education. The program coordinator suggests that this success rate 
could be even higher if not for the inclusion of "heavy-duty" special education students 
(e.g., Downs syndrome) who comprise some 25 percent of the program's trainees. To 
accommodate these students, training is conducted over longer periods of time, 
placements tend to be in hospital service areas that do not provide direct patient care 
(e.g., food services), and students in the program work with one of the district's "Special 
Education Technicians." 



FACILITIES AND EQUIPMENT 



ERLC 



The facility for Health Occupations, including the Nursing assistant program, consists of a 
large "portable classroom" facility. There are three large classrooms, a nursing lab (with 
beds, etc.), a clinical lab area (with microscopes and other equipment), a physical therapy 
area (with mirrors, mats, parallel bars, etc.), and a fully-equipped simulated dental/doctor's 
office. The facility also has a kitchen area with stove and refrigerator, several restrooms, 
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offices, and storage areas, and open space. The facility appears extremely well-equipped 
for its purposes, and does simulate environments that would be found in a medical clinic, 
laboratory, or hospital. 

According to the project coordinator, ADE funds played a considerable role in allowing the 
program to evolve. In particular, the coordinator said that virtually all of the facility's state 
of-the-art equipment was purchased through ADE grunt monies. Nevertheless, the 
district's financial commitment to the program has also been substantial, and the 
coordinator says that even without ADE funds, "I think we would have found a way to 
develop [the program]." 

TEACHER QUALIFICATIONS 

Nine staff teach Level I courses in the three middle schools. All are certified in their 
respective areas: computer instructors are business certified; the technology teachers are 
industrial education certified; and the life management staff are certified in home 
economics. Two instructors teach the Medical Arts course, one at each high school. They 
are certified in science and additionally certified or endorsed in vocational education. Two 
registered nurses teach the Level III programs; the program coordinator is also a registered 
nurse. One instructor is also a certified science and physical education teacher; another is 
a practicing nurse practitioner in family medicine; the program coordinator is certified in 
rehabilitation. 

Professional development activities are diverse and ongoing. There are five annual 
inservice days each year, in addition to conferences and seminars. Moreover, staff 
regularly serve on a variety of district and state committees (e.g., curriculum committees). 
All maintain active connections with the health care industry. 

COMMENTS 

Program documentation is informative, extensive, and thorough. Among the materials the 
evaluator had the opportunity to examine were a curriculum materials review form, 
competency lists, certificates of completion, advisory committee lists and committee 
recruitment materials, student and community "fact sheets," course syllabi, articulation 
agreements, the district statement of philosophy, and the "Health Occupations 5-year 
Plan." Based on a review of these materials and the site visit, the evaluator's overall 
impression of Deer Valley's Nurse Assisting program is that it provides a high quality 
training experience for students choosing this health occupation. Staff commitment is 
evident and students, although only spoken to briefly, voiced strong support for the 
program. The facilities are impressive. 

Asked what are the key strengths of the Nursing Assistant program, the coordinator 
mentioned an "exceptional" teaching staff and close cooperation and coordination between 
each level of the curriculum. These strengths seem apparent and prompted the evaluator 
to question the results of the initial program survey, in which the principal responded not 
having a "model" Level I or Level II program. 
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Following ADE-criteria for a fully implemented model program, the reason that Deer 
Valley's Level program might not be considered a "model" is because all students might 
not receive a full year of instruction (or the equivalent of 225 minutes per week) and there 
are no guarantees that 100 percent of students in grades seven through nine will have 
taken a Level i course due to new students/transfers entering the district. Similarly, Level II 
cannot guarantee that all students will have completed a Level I program, thus excluding it 
from meeting "model M criteria. 

Apart from these "deviations" from the ADE criteria, if "modelness" equals "quality," then 
Deer Valley does appear to represent a model program. The program is well organized. All 
instructional and curricula components are in place and aligned, accommodating the 
special needs of IVEP students. Facilities simulate business-industry standards, and provide 
students with hands-on and occupational experiences. Faculty are fully and appropriately 
certified for the respective levels of instruction, and there is evidence Of strong local 
support for the program. 
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DEER VALLEY VTECH CENTER, Deer Valley Unified: Health Occupations -Nursing Assistant 



Program Area and Characteristics 



LEVEL 
I II III 



yes 



yes 



yes 



no 



no 



yes 



no 



yes 



yes 



I. ORGANIZATION 

1 . Instruction provides: 

• an exploration/skill development for all occupations. 

• skill development for a cluster of related occupations. 

• skill development for specific occupation(s). 

2. Students served: 

• 100% in the targeted grade level (7-9). 

• In the targeted grade levels (9-10) have previously completed a Level I 
experience. 

• Students have the opportunity to participate in more than one cluster, if 
desired. 

• In the targeted grade levels (11-12) have completed Level I and ll 
experiences. 

3. The experience is a minimum of: 

• one year in length or equivalent to 225 min/week. 

• 360 hours in length. 

4. The experience provides a coherent sequence of instruction: 

• required for ali vocational technological programs. 

• each cluster leads to at least one specific Level III experience. 

• leads to employment and advanced training. 

5. The experience has the support of the local school/district administration. 

6. An active advisory council assists in designing and implementing this 
experience. 

II. CURRICULUM 

7. Curriculum is competency-based with each student's attainment of each 
competency monitored. 

8. Curriculum is designed to deliver all academic and occupational 
competencies (approved by ADE) in: 

• the draft Level I Framework. 

• at least one cluster. 

• specific occupation(s). 



yes 



yes 

yes 

yes 

yes yes yes 
yes yes yes 

yes yes yes 



yes 



yes 



yes 
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Program Area and Characteristics LEVEL 

1 II HI 

II. CURRICULUM - continued 

9. Curriculum includes all six of the Model "Strands." yes yes yes 

10. Curriculum provides for student leadership development, which can include 

the use of vocational student organizations. no yes yes 

III. INSTRUCTIONAL DELIVERY 

11. The instruction provides students with experiences in: 

• a variety of occupational areas. yes 

• occupational areas associated with the particular ciuster(s). yes 

• specific occupation(s). yes 

12. The teacher serves as a facilitator of instruction, managing a variety of 

learning experiences for students. yes yes yes 

13. Students work in groups or teams frequently. yes yes yes 

14. Instructional activities include industry-realistic situations and materials, and 

are evaluated according to industry-related standards. yes yes yes 

15. The instructional delivery includes occupational experiences which can 

include shadowing and cooperative education. yes yes yes 

IV. FACILITIES AND EQUIPMENT 

16. The instructional facility provides a multi-purpose environment with 

appropriate technology and support systems. yes yes yes 

17. The environment simulates one that would be found in business and 

industry. yes yes yes 

18. The equipment and materials utilized in the experience are determined on 

the basis of the curriculum and industry standards. yes yes yes 

V. TEACHER QUALIFICATIONS 

19. The teacher(s) are vocationally certified in: 

• vocational areas or if a team of teachers instruct the Level I experience, at 

least one will be vocationally certified. yes 

• aDpropriate vocational areas related to the specific cluster content. yes 

• the specific vocational area. yes 

20. The teacher(s) have recent inservice to update and upgrade their teaching 

techniques and content knowledge. yes yes yes 



ERIC 
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Site: COCONINO HIGH SCHOOL, Flagstaff Unified School District 
Program: Trade and Industry — Electronics 



SETTING 

The Trade and Industry Electronics program at Coconino High School was selected as a 
rural, self-decribed "non-comprehensive" program that has received "Model site" funding 
from ADE for three years. The school is located in northern Arizona, Coconino County, in 
an area that encompasses private and government held property. It is a popular resort area 
with a stable school population. 

METHODS 

Data were collected primarily from two sources— the electronics instructor and the District 
Vocational Director, A tour of the facility was given by the instructor, who was also in the 
midst of preparation for a community open house that same day and the following 
evening. Documentation reviewed, but not given to the interviewer, included: 

1 . Course syllabi 

2. District Vocational Program Guidelines/Standards 

3. Instructional unit: Electricity 

4. Tech Lab manuals (4) 

5. Advisory Committee minutes 

This report summarizes the interviews with the two parties. 
ORGANIZATION 

Levels N/i 

The district has no Level I classes but does offer a course titled "Tech Lab" which is 
available to students in grades 9 and 10. It is a one-semester course in which students 
rotate through the various lab stations. The curriculum is matched to levels and clusters. 
The course is not required of all students, but is encouraged as a prerequisite to the Level 
II and III program. The curricuum explores manufacturing, construction and robotics. 

The high school offers introductory level programs in accounting/computing occupations, 
administrative support, health, nursing, building trades, communication/electronics, 
computer electronics and auto mechanics. The district does not consider these to be Level 
II programs. Typical courses available include Woods, Drafting, General Business and 
Introduction to Electronics. 

Level III programs, for students in grades 10-12, provide students with specific 
occupational training. The district does not consider its programs to be Level III Model 
programs. They offer courses in Accounting/Computing Occupations, Administrative 
Support, Health, Nursing, Building Trades, Comm/Electronics, Computer Electronics and 
Auto Mechanics. 
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Students are encouraged to complete two particular courses as prerequisites to the 
Electronics Program. They may take one semester of Tech Lab, one semester of 
Introduction to Electronics or a full year of Introduction to Electronics. 

The electronics program utilizes self-developed materials in addition to those from the Tech 
Lab (Colorado) as resources. The technical manuals in use are from L. J. Technical 
Systems (Digiac). They are competency-based materials. 

VSOs are available to students at Coconino High School. Specifically, students in the 
electronics program are encouraged to participate in VICA. The school also offers a 
cooperative education experience for those students desiring it. 

Local and District Support 

Staff have rated overall district support for vocational programs at 4 on a scale of 10, with 
10 being high. Support for the electronics program was rated at 6. When queried about 
the fairly low rating, the staff expressed that "our programs do not have much priority in 
the district/campus. We are not seen as an integral part of the educational process." 

Students and counseling staff are encouraged to develop a four-year plan for high school. 
These plans are not mandatory and vocational staff believe that little emphasis is given to 
vocational education electives and student career goals. 

Advisory Council 

Several campus-based curriculum committees are operational. In addition, the District has 
a 14 member Vocational Education Advisory Committee. The committee meets four times 
a year to review new programs, equipment needs, facilities and so forth. The technology 
classrooms at Coconino High School have been designated as a demonstration site for the 
district. As such, the site continues to receive support and attention from the Advisory 
Committee. 

CURRICULUM 

As noted earlier, the Technology Lab utilizes both teacher developed and commercially 
developed materials (L. J. Technical Systems). When asked whether or not the six strands 
were used, the staff indicated that they were but only as they existed in the present 
curriculum and instructional units. They felt that the Technology Systems materials 
probably exceeded the competencies contained in the ADE Competencies. 

Students utilize computerized trainers; modules are self-starting and self-paced (open 
entry/exit). The instructor works with the special education resource class on units on 
electricity. 

Teacher assessment of competency attainment is based on observation and performance. 
Competencies are measured on an individual basis as each student completes a particular 
project. The majority of assessment activities are centered on production using the Tech 
Lab work stations. 
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The district conducts a follow-up of completers one year after graduation. Response rates 
have not been particularly high, but more than half of those responding are continuing in 
school. 



INSTRUCTIONAL DELIVERY 



Instructional delivery is concentrated on individual and small group projects utilizing the 
Tech Lab and electronics specialized equipment. Hands-on projects and production projects 
are emphasized. The instructor also utilizes appropriate concepts from mathematics and 
science to integrate instruction. 



IVEP Students 



The instructor works with the resource teacher particularly on units in electricity. In 
addition, special needs students who may be enrolled in the electronics program are 
provided individual assistance as needed. 



FACILITIES AND EQUIPMENT 



Coconino High School is a relatively new school plant. The District has upgraded several 
areas in the facility, including the Technology Lab. The instructional environment in the lab 
easily meets or exceeds that of industry in terms of equipment. The instructor has 
indicated that software needs are "greater than my equipment needs." Some of the 
vocational areas have space needs and the Vocational Director has indicated that "we 
always need dollars for curriculum development efforts." 

TEACHER QUALIFICATIONS 



All teachers at Coconino High School teaching in vocational areas are certified with 
appropriate secondary and vocational endorsements. 

District funds have been used to support vocational teacher participation in the Vocational 
Summer Conference and Curriculum Dissemination Workshops. 



COMMENTS 
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Coconino High School has an exemplary program in the Tech Lab/electronics area. This 
program is not part of a comprehensive Level III program and does not meet state criteria 
for one. On the other hand, the electronics program has great potential to be a front runner 
for a Level III program. Assuming that the school is able to better define and implement a 
sequence of courses, it could easily become a model site in electronics. 

There are some problems related to scheduling, facility upgrades in some areas, and dollars 
to support curriculum and professional development activities. The opportunity exists to 
develop more cohesive vocational programs reaching greater numbers of students. 
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COCONINO HIGH SCHOOL, Flagstaff Unified: Trade & Industry- Electronics 



Program Area and Characteristics 



I. ORGANIZATION 

1 . Instruction provides: 

• an exploration/skill development for all occupations. 

• skill development for a cluster of relaxed occupations. 

• skill development for specific occupation(s). 

2. Students served: 

• 100% in the targeted grade level (7-9). 

• In the targeted grade levels (9-10) have previously completed a Level I 
experience. 

• Students have the opportunity to participate in more than one cluster, if 
desired. 

• In the targeted grade levels (11-12) have completed Level I and II 
experiences. 

3. The experience is a minimum of: 

• one year in length or equivalent to 225 min/week. 

• 360 hours in length. 

4. The experience provides a coherent sequence of instruction: 

• required for all vocational technological programs. 

• each cluster leads to at least one specific Level III experience. 

• leads to employment and advanced training. 

5. The experience has the support of the local school/district administration. 



LEVEL 
I II III 



An active advisory council assists in designing and implementing this 
experience. 



II. CURRICULUM 

7. Curriculum is competency-based with each student's attainment of each 
competency monitored. 

8. Curriculum is designed to deliver all academic and occupational 
competencies (approved by ADE) in: 

• the draft Level I Framework. 

• at least one cluster. 

• specific occupation(s). 



no 



no 



no 



no 



yes 



no 



yes 



no ns 



no 



yes 



no 



yes 



yes 

no ns yes 
no yes yes 

no ns yes 



no 



yes 
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Program Area and Characteristics LEVEL 
I II III 

II. CURRICULUM - continued 

9. Curriculum includes all six of the Model "Strands." no no no 

10. Curriculum provides for student leadership development, which can include 

the use of vocational student organizations. no yes yes 

III. INSTRUCTIONAL DELIVERY 

1 1 . The instruction provides students with experiences in: 

• a variety of occupational areas. yes 

• occupational areas associated with the particular cluster(s). yes 

• specific occupation(s). yes 

12. The teacher serves as a facilitator of instruction, managing a variety of 

learning experiences for students. ns ns yes 

13. Students work in groups or teams frequently. ns ns yes 

14. Instructional activities include industry-realistic situations and materials, and 

are evaluated according to industry-related standards. no no yes 

15. The instructional delivery includes occupational experiences which can 

include shadowing and cooperative education. no nc yes 

IV. FACILITIES AND EQUIPMENT 

1 6. The instructional facility provides a multi-purpose environment with 

appropriate technology and support systems. no yes yes 

1 7. The environment simulates one that would be found in business and 

industry. no yes yes 

18. The equipment and materials utilized in the experience are determined on 

the basis of the curriculum and industry standards. no yes yes 

V. TEACHER QUALIFICATIONS 

19. The teacher(s) are vocationally certified in: 

• vocational areas or if a team of teachers instruct the Level I experience, at 

least one will be vocationally certified. yes 

• appropriate vocational areas related to the specific cluster content. ye 

• the specific vocational area. yes 

20. The teacher(s) have recent inservice to update and upgrade their teaching 

techniques and content knowledge. yes yes yes 



ERLC 



B-52 

133 



Site: VALLEY UNION HIGH SCHOOL, Valley Union High School District 
Program: Agriculture— Business and Management 



SETTING 

Valley Union High School was selected to represent a self-reported "non-comprehensive" 
programs that received "Model site" funding for two years. The high school is located in a 
remote rural area of Southeastern Arizona. It has five feeder elementary schools. The 
entire Agriculture program (including both Ag Business and Management and Agricultural 
Mechanics) is taught by one instructor who has been with the district for 30+ years. 

METHOD 

This report is based on data collected via an interview with the agriculture teacher, a tour 
of the facility, and documents provided by the instructor. A brief, informal interview was 
held with the principal after the interview with the teacher. Most of the information in this 
report was provided by the agriculture teacher. 

ORGANIZATION 

Level I 

A Level I program is not offered at the school. Site staff believe that a Level 1 course 
should be available to students in the eighth grade. The elementary schools feeding Valley 
Union High are very small, however, sending between eight and 29 students to Valley 
Union each year. Their limited budgets restrict program development and inhibit their 
ability to fund additional faculty to teach a Level 1 course. Furthermore, collaborative 
mechanisms do not exist either among elementary schools or between the elementary 
schools and Valley Union. 

Level II 

The Level II component of the program at this site is comprised of the two introductory 
courses, Vocational Agriculture I and Vocational Agriculture II. These courses provide a 
basic introduction to plant and animal science and are considered core curriculum for any 
student pursuing a program in agriculture. These are orso year courses offered as electives 
available to any student in grades 9 through 12. Most students take the courses in the 
ninth and tenth grades. Vocational Agriculture I and II are prerequisites for Vocational 
Agriculture III and IV. 

Level III 

Vocational Agriculture III and IV comprise the Level III program at Valley Union High 
School. The Agricultural Mechanics program is incorporated in these courses. These are 
one year courses offered in alternate years. Classes are small, allowing for highly 
individualized instruction. Many students participate in a Supervised Occupational 
Experience (S.O.E.). Limited work experience opportunities in this geographical region 
mean that most students participate in S.O.E.s on weekends and during the summer. 
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Students may prepare livestock for the county fair, for axample, as part of a qualified 
S.O.E. The S.O.E.s are coordinated by the agriculture teacher, who is a 12-month 
employee of the school district. 

Local and District Support 

The teacher ranked district support on a scale of one to ten as a "10" (with 10 being 
high). He said: "I've never really been turned down flat for anything. ...The Board is really 
supportive." 

Advisory Council 

Because of the small size of the school and the surrounding community, the advisory 
group for agriculture is the same as the school district's advisory group. It is made up of 
local business people (mostly ranchers and farmers), a teacher, and parents. The group 
acts as a sounding board for community and school concerns. The group is not actively 
involved in vocational education issues. 

CURRICULUM 

When asked about the ADE curriculum strands, the teacher responded, "I've been using 
what we developed, which is an Agriculture core curriculum. I don't know if it fits in with 
the strands or not. It's the core curriculum that was developed by the state 
department.. ..That's what I use." 

Although he utilizes the state's core curriculum, he also stated that he felt that, even 
though the state was attempting to "standardize everybody," he believed that local 
curriculum needs must still be addressed and that the state's program "could not replace 
all that." However, he did see the value in common curriculum as well: "But there are 
things they ought to know wherever they (students) are. Because they won't stay here." 

The instructor provided a checklist, the Vocational Agriculture Core Curriculum 
Occupational Readiness Record, which illustrates the scope of the core curriculum. 

Pretesting and posttesting are administered to all Agriculture students to measure growth 
in academic skills. The test addresses basic reading and mathematics skills. In regard to 
occupational competencies, the teacher said: "I guess I evaluate them myself. We had a 
checklist, that wasn't consistent either. So I just, more or less, some of these 
competencies, if they can do it, I just evaluate it in my head. I don't keep a written 
record." 

The teacher bases student grading predominantly on his own judgment of their progress 
and puts little weight on the results of the pretesting and posttesting. The test was 
developed in the Mesa School District. Scores were used for completion of the 
Performance Standards Report. The teacher spoke of the value of the test results: "It (the 
test) would not determine whether they pass or fail." 
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INSTRUCTIONAL DELIVERY 



The instructional delivery system varies with the unit of instruction. More traditional 
classroom methods are used when topics such as parliamentary procedure are introduced. 
By way of contrast., instruction of such subjects as computerized recordkeeping and 
introductory accounting is more individualized. The balance of the instruction is project- 
oriented. Students, depending on their level of competence, may work either individually or 
in groups. 

Many of the units are taught hand-in-hand with the students' involvement in FFA, which 
affords them the opportunity to work on local farms and ranches as well as at the school 
site. At the time of observations for this report, students were constructing their own 
greenhodse as part of their program. The teacher stated that he uses FFA as "an incentive 
for a lot of our classroom work, especially leadership, public speaking. ..I found that those 
who are not active in the FFA don't get the full benefit of the Ag. program. They can' I see 
the depth of it." 

The FFA chapter at Valley Union has produced several state officers. Many of the Valley 
Union FFA members have won state awards in public speaking. Close to 100% of the 
Agriculture students participate in FFA, and many stay involved after g.aduation. Part of 
the Ag. I curriculum is dedicated to FFA topics including FFA Organization, Public 
Speaking, and Parliamentary Procedure. 

IVEP Students 

Some of the IVEP students enrolled in this program, particularly LEP students, are assisted 
by a school district aide. Those with reading difficulties may be given more time to 
complete some projects. The teacher collaborates with the special education instructors on 
a regular basis. Although he may adjust standards for IVEP students, the instructor said "I 
don't lower the expectations." Because of the hands-on nature of the classes, IVEP 
students have considerable success in this program. IVEP students are expected to take 
the same tests as all other students. 

FACILITIES AND EQUIPMENT 

Program facilities are identified as one of the program's major strengths. The facility is 
comprised of a classroom with approximately 18 computer stations that are linked into the 
school network. Students are exposed to much of the business aspect of agri-business 
through this computer network. This laboratory was funded by an ADE grant. Although the 
up-to-date technology was appreciated by both the district and the teacher, it was also 
viewed by the Agriculture teacher as a "burden" because of the increased demands it 
made on him to learn the technology. "I had to do a lot of learning too." As the sole 
instructor of the agriculture programs, his efforts to stay current are limited by the free 
time he has available. The movement toward scientifically based programs, i.e., the 
biological systems approach to agriculture, has also placed new demands on this teacher 
of 30 years. 
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Some frustration was expressed that the ADE grant for equipment acquisition was not 
accompanied by more technical support. The instructor felt that the school's relative lack 
of technical expertise led them to purchase some things they did not need and to overlook 
some that they did. 

In addition to the computer lab, facilities at Valley Union include a large work station area 
where students have access to numerous types of welding equipment. 

TEACHER QUALIFICATIONS 

The teacher is a certified Vocational Agriculture teacher. He participates regularly in state- 
sponsored training related to agriculture education. The district has limited professional 
development programs, offering none during the 1993-94 school year. He is given released 
time in the summer to attend workshops, particularly at the University of Arizona. He 
described the programs at this univeristy as "excellent." 

COMMENTS 

Multiple factors come into play when considering this district's inability to implement a 
more comprehensive program in Agriculture — Business Management. The most salient 
point seems to be the lack of available professional development time for the instructor. 
The instructor is the sole teacher in the department and has no Vocational Director (the 
principal is the director in title but not function). He is responsible not only for teaching all 
sections of Agriculture, but also for heading the FFA, coordinating the Supervised 
Occupational Experiences, writing the curriculum, completing the Performance Standards 
reports, and so on. 

Another factor limiting the district is the lack of district influence over decisions made at 
the feeder elementary schools. Without district influence, elementary schools are unlikely 
to develop Level 1 programs. Valley's Level II program is somewhat representative of the 
cluster concept expected by ADE. However, there are some limitations that have 
prohibited the full development of a Level II program. 

In sum, there seem to be expectations to implement a comprehensive program, yet barriers 
that individuals and districts face to meet these expectations have not been fully 
addressed. 
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VALLEY UNION HIGH SCHOOL, Valley UHSD: Agriculture-Business & Management 



Program Area and Characteristics 



LEVEL 
I II III 



no 



no 



no 



no 



An active advisory council assists in designing and implementing this 
experience. 



II. CURRICULUM 

7. Curriculum is competency-based with each student's attainment of each 
competency monitored. 

8. Curriculum is designed to deliver all academic and occupational 
competencies (approved by ADE) in: 

• the draft Level I Framework. 

• at least one cluster. 

• specific occupation(s). 



no 



no 



no 



no 



no 



yes 



I. ORGANIZATION 

1 . Instruction provides: 

• an exploration/skill development for all occupations. 

• skill development for a cluster of related occupations. 
*> skill development for specific occupation(s). 

2. Students served: 

• 100% in the targeted grade level (7-9). 

• In the targeted grade levels {9-1 C) have previously completed a Level I 
experience. 

• Students have the opportunity to participate in more than one cluster, if 
desired. 

• In the targeted grade levels (1 1-12) have completed Level I and i! 
experiences. 

3. The experience is a minimum of: 

• one year in length or equivalent to 225 min/week. 

• 360 hours in length. 

4. The experience provides a coherent sequence of instruction: 

• required for all vocational technological programs. 

• each cluster leads to at least one specific Level ill experience. 

• leads to employment and advanced training. 

5. The experience has the support of the local school/district administration. 



no 



ns 



yes 



yes 



yes 



yes 

yes 

yes yes 
no no 

ns ns 



yes 



yes 
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Program Area and Characteristics t II III 

It. CURRICULUM - continued 

9. Curriculum includes ail six of the Model "Strands." no no no 

10. Curriculum provides for student leadership development, which can include 

the use of vocational student organizations. no yes yes 

ill. INSTRUCTIONAL DELIVERY 

11. The instruction provides students with experiences in: 

• a variety of occupational areas. no 

• occupational areas associated with the particular cluster(s). yes 

• specific occupation(s). yes 

12. The teacher serves as a facilitator of instruction, managing a variety of 

learning experiences for students. no yes yes 

13. Students work in groups or teams frequently. no yes yes 

14. Instructional activities include industry-realistic situations and materials, and 

are evaluated according to industry-related standards. no ns yes 

15. The instructional delivery includes occupational experiences which can 

include shadowing and cooperative education. no yes yes 

IV. FACILITIES AND EQUIPMENT 

16. The instructional facility provides a multi-purpose environment with 

appropriate technology and support systems. no yes yes 

17. The environment simulates one that would be found in business and 

industry. no yes yes 

18. The equipment and materials utilized in the experience are determined on 

the basis of the curriculum and industry standards. no no no 

V. TEACHER QUALIFICATIONS 

19. The teacher(s) are vocationally certified in: 

• vocational areas or if a team of teachers instruct the Level I experience, at 

least one will be vocationally certified. no 

• appropriate vocational areas related to the specific cluster content. yes 

• the specific vocational area. yes 

20. The teacher(s) have recent inservice to update and upgrade their teaching 

techniques and content knowledge. no yes yes 
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Site: KOFA HIGH SCHOOL, Yuma Union High School District 
Program: Trade and Industry— Drafting 



SETTING 

The drafting program at Kofa High School was selected as representative of a Trades and 
Industry program at an urban/metropolitan, self-decribed "non-comprehensive site," that 
has never received additional state "Model site" funding. Kofa is one of four schools in the 
Yuma Union High School District. 

METHOD 

Data collection was accomplished through an audiotaped interview with the district 
Vocational Education Director and the District Curriculum Coordinator and an audiotaped 
interview with the drafting teacher. The researcher also videotaped a tour of facilities 
taken with the drafting teacher. Program documents were provided by the Vocational 
Director and the instructor. 

ORGANIZATION 

Level I 

An introductory course titled Exploratory Technology is offered to students at Kofa. This 
course provides instruction for one quarter in each of the following: drafting, 
woodworking, metalworking and electronics. This course does not meet ADE Level 1 
criteria. 

The district is working towards the creation of articulated Level 1 programs in junior high 
schools feeding Kofa. With facilitation provided by the district, communication and 
articulation between elementary and high school districts has begun. The district 
vocational director acknowledges "it's a slow process; it's not something we just walk in 
and do overnight. ...Our dream is to have the junior highs prep these kids; they come in to 
ours in the ninth grade and we move them on. Right now we have some duplication of 
effort." Some, he says, are "in the breadbox mode and some are into high tech." 
Implementing the "Lab 2000" concept in feeder schools, he felt, "would prepare them 
(students) to come into our technical areas." 

Concern was expressed that the fiscal process, which channels funding through the high 
schools to the feeder schools, has made the development of Level 1 programs more 
cumbersome. Cooperation and communication among schools has improved, but funding 
remains a barrier to future expansion. 

Level II 

Level II, at this site, is typically defined across most of the vocational areas as the first 
course in the sequence, i.e., Construction Trades I, Building Trades I, Automotive 
Technology I, and so on. Drafting Technology I is the first course in a two-year sequence, 
and is considered the Level II course in the drafting area. Courses at this level are 
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described as the "prep" courses which "get kids ready for the Tech (Level III)." The 
exploratory course "flows into Drafting I and Drafting I flows into Drafting II." To explore 
multiple areas, students must enroll in a variety of year-long courses. The instruction is 
neither clustered nor exploratory. 

Level III 

Level III consists of courses that provide students with advanced level training in specific 
occupational areas. Drafting Technology II is the Level III course in drafting. The course 
description describes the course as "a study of advanced drafting practices...." The course 
is an extension of its Level II prerequisite, Drafting Technology !. 

Local and District Support 

The district Curriculum Coordinator described herself as a "sounding board" who 
influences coordination and communication among the curricuiar areas. She is the one who 
monitors the "big picture." Those who are heads of the specialty areas (e.g., special 
education, vocational education, Chapter I) are called "facilitators." The vocational 
facilitator described his relationship with the district as very positive (a "10" on a scale of 
1 to 10 with 10 being high). "We're allowed the flexibility to create," he reported. He 
indicated that the relationship with the school board is positive; two board members are 
involved in the Performance Standards evaluation team. Efforts have been made at the 
building level to improve communication among principals regarding the different programs 
offered at each of the different high schools. The vocational facilitator described the 
principals as "supportive". 

Advisory Council 

The drafting advisory committee is district-wide and includes membership from business, 
industry and education. Educators from public schools and from Arizona Western College 
are involved. Any proposed curricui'Jiii or equipment changes are first examined by the 
advisory committee. The committee is very active both instrumentally and politically. They 
are advocates for the vocational program in the school and in the community. 

CURRICULUM 

VSOs are not an estabiished part of the curriculum and student participation in programs 
such as VICA, FFA, HERO and FBLA is voluntary. They are, however, considered to be 
vehicles for teaching ADE "curriculum strands". 

All courses have a written scope and sequence. Competency indicators for Drafting 
Technology I (Level I!) and Drafting Technology II (Level III) were provided as examples of 
the scope and sequence for these courses. Records are kept on each student. Teacher 
discretion prevails in determining appropriate methods and standards of assessment for 
individual students. Additionally, district staff are working to integrate academic and 
vocational instruction in a curriculum designed to support six high school exit skills which 
are consistent with state Essential Skills. Over the next four years, plans are to determine 
the linkages among subject areas (e.g., the commonalities in language arts, social studies, 
mathematics, and technology) and vo develop more integrated methods of assessment of 
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those skills across areas. They hope to minimize duplication and streamline instructional 
and assessment procedures. In addition, the local school board is currently considering 
awarding academic credit for some vocational courses. 

The ability of staff to incorporate a career development component of curriculum is 
restricted by two factors. First, the geographical location of this school limits students' 
access to industry-based experience. Second, programs available at Arizona Western 
College are limited in scope. 



INSTRUCTIONAL DELIVERY 



The vocational director described the drafting instructor as outstanding, with in-depth 
industry knowledge. He recently received a state award for teaching excellence. Working 
with limited facilities, he has developed a well-funded program, utilizing his close contacts 
with local industry to design and implement curriculum. The district vocational director 
noted "it's only a small room. He can put a gallon in a quart jar." Classroom observation 
revealed that students' work is primarily project-oriented. 



IVEP Students 



Services for iVEP students are coordinated through the vocational coordinator with 
facilitators from other areas such as Migrant Education, Special Education, and Guidance 
and Counseling. In relation to measurement of competency for IVEP students, the director 
stated that "we strive to have every student obtain these competencies.. ..We adjust how 
we teach them. Every child can succeed." 



FACILITIES AND EQUIPMENT 



The vocational director describes equipment as satisfactory, but the facility as very limited. 
The school's limited supply of computers are of varying age and capacity, causing 
instructional difficulties. The entire Drafting Technology program is housed in one small 
classroom with computer stations around the perimeter of the room and drafting tables in 
the center. An effort is being made to pass a bond issue which will provide for a new 
technology that should solve the current facility constraints. 



TEACHER QUALIFICATIONS 
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The drafting instructor is fully certified in his instructional area and has extensive industry- 
relevant work experience. The district and the vocational director encourage all teachers to 
participate in state-sponsored workshops and institutes. Teachers are much more 
amenable to attending training during the school year but are less likely to participate 
during the summer. Many are in college courses that conflict with summer inservice 
opportunities. The district offers regular professional development activities on topics such 
as stress management, wellness, and Spanish. 
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COMMENTS 



The Kofa program has substantial support at the district and community level. However, 
funding from the state directly to the feeder districts could help facilitate the development 
of a Level I program. Their Level II programs are specialized introductory courses rather 
than exploratory clusters. These courses are limited to specific subject areas. In this 
regard, the district programs do not follow the suggested ADE criteria for Level II. Both 
Levels I and II, as currently conceptualized by the vocational coordinator and the high 
school teacher, are introductory or preparatory courses for Level III. 
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KOFA HIGH SCHOOL, Yuma UHSD: Trade & Industry -Drafting 



Program Area and Characteristics 



LEVEL 
I II 



no 



no 



yes 



no 



I. ORGANIZATION 

1. Instruction provides: 

• an exploration/skill development for all occupations. 

• skill development for a cluster of related occupations. 

• skill development for specific occupation(s). 

2. Students served: 

• 100% in the targeted grade level (7-9). 

• In the targeted grade levels (9-10) have previously completed a Level I 
experience. 

• Students have the opportunity to participate in more than one cluster, if 
desired. 

• In the targeted grade levels (1 1-12) have completed Level I and II 
experiences. 

3. The experience is a minimum of: 

• one year in length or equivalent to 225 min/week. 

• 360 hours in length. 

4. The experience provides a coherent sequence of instruction: 

• required for all vocational technological programs. 

• each cluster leads to at least one specific Level III experience. 

• leads to employment and advanced training. 

5. The experience has the support of the local school/district administration. 



no 



yes 



no 



no yes 



yes 



An active advisory council assists in designing and implementing this 
experience. 



II. CURRICULUM 

7. Curriculum is competency-based with each student's attainment of each 
competency monitored. 

8. Curriculum is designed to deliver all academic and occupational 
competencies (approved by ADE) in: 

• the draft Level I Framework. 

• at least one cluster. 

• specific occupation(s). 



no 

yes 

yes 

yes yes yes 
no yes yes 



ns yes yes 



no 



yes 



yes 
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Program Area and Characteristics LEVEL 
I 1) HI 

II. CURRICULUM - continued 

9. Curriculum includes all six of the Model "Strands." no no no 

10. Curriculum provides for student leadership development, which can include 

the use of vocational student organizations. no yes yes 

IN. INSTRUCTIONAL DELIVERY 

1 1 . The instruction provides students with experiences in: 

• a variety of occupational areas. no 

• occupational areas associated with the particular cluster(s). no 

• specific occupation(s). yes 

12. The teacher serves as a facilitator of instruction, managing a variety of 

learning experiences for students. yes yes yes 

13. Students work in groups or teams frequently. yes ns yes 

14. Instructional activities include industry-realistic situations and materials, and 

are evaluated according to industry-related standards. ns no no 

15. The instructional delivery includes occupational experiences which can 

include shadowing and cooperative education. ns ns yes 

IV. FACILITIES AND EQUIPMENT 

16. The instructional facility provides a multi-purpose environment with 

appropriate technology and support systems. yes no no 

17. The environment simulates one that would be found in business and 

industry. ns no no 

18. The equipment and materials utilized in the experience are determined on 

the basis of the curriculum and industry standards. ns yes yes 

V. TEACHER QUALIFICATIONS 

19. The teacher(s) are vocationally certified in: 

• vocational areas or if a team of teachers instruct the Level I experience, at 

least one will be vocationally certified. no 

• appropriate vocational areas related to the specific cluster content. ves 

• the specific vocational area. yes 

20. The teacher(s) have recent inservice to uodate and upgrade their teaching 

techniques and content knowledge. yes yes yes 
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Site: APACHE JUNCTION HIGH SCHOOL, Apache Junction Unified School 
District 

Program: Home Economics — Food Production 
SETTING 

Home Economics programs at Apache Junction High School were selected to represent a 
non-comprehensive, non-funded program. The high school is located in Pinal County in an 
area that is designated as rural but abuts the larger community of Mesa to the west. It 
aiso is located in an area that is at the county line between Maricopa and Pinal County. 
The area has a high winter visitor population, but a fairly stable school population. 

METHOD 

Staff interviews were the primary data-collection method. A comprehensive interview was 
held with the District Vocational Coordinator. Less extensive interviews were < 'so held 
with the Home Economics and Business Education teachers, the instructional aide and the 
school principal. A tour of the facility was provided by the Vocational Coordinator. 

Documentation provided for review, but not given to the interviewer, included: 

1. District curriculum guides 

2. Board Policy Statement/Vocational Education 

3. Course description booklet 

4. SIMS (instructional management system) 

5. Articulation Committee minutes 

This report summarizes the interviews, tour and documentation materials. 
ORGANIZATION 

Levels Nil 

The district does not consider that any of its programs meet the ADE criterion as a Level I 
program. It does have "bits and pieces" in place through several courses, including home 
economics, success skills and shop. 

A one-semester course titled "Success Skills" is required for students in ninth grade. The 
district plans to have this course become a requirement for students in grades 7-8 as well. 
It presently utilizes four of the six ADE model strands (thinking skills, career development, 
life management and applied academics) and is taught by a vocationally certificated 
teacher. 

Level II programs are not offered at Apache Junction. Rather, separate courses more 
traditional in nature are offered. They include Home Economics (Foods 1 and2), Housing 
and Interior Design, Human Relations (1 and 2), Child Development (1 and 2} and Fashion 
and Fabric. These courses are one semester in length. 
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Level III programs meeting ADE criteria are offered in Accounting/Computing, 
Administrative Support, Food Production Management, Carpentry, Auto Mechanics, 
Drafting, and Welding. In the Home Economics program, there are several classrooms 
which are traditional, but have computers available. The Foods area is fairly well equipped. 

VSOs are available to provide leadership opportunities for vocational students, including 
FBLA, DECA, STRIVE and a COE club. Other growth experiences are provided through field 
visits and visiting lecturers. 

The programs utilize career files and the SIMS reporting system. This system provides a 
printout of attained competencies. The district conducts a student follow up by using a 
checklist with last known address as the pojnt of contact. Follow-up is conducted early in 
December, followed by a second contact in February and a telephone follow-up in April. 

Local and District Support 

District support for vocational programs was rated at 7-8 on a scale of 10 with 10 being 
highest. The staff expressed that there was a high level of external influences (politics) 
that affect the attitude toward vocational education. There was also some concern that 
the district "can't keep up with changes and teachers are bogged down with paperwork." 
There is a board policy statement regarding vocational education. 

Advisory Council 

There is an overall District Advisory Committee for vocational education. The committee 
meets four times a year, sometimes more. There is broad representation from the 
community and an average of eight to nine persons attend meetings regularly. District 
representatives feel that there is strong commitment from the advisory committee 
members. ^ 

In addition to the advisory committee, the district has a consultant contract for technical 
assistance from an external consultant. The district also participates in articulation 
committees with Central Arizona College and is part of the EVIT Tech Prep consortium. 

CURRICULUM 

The Apache Junction district has curriculum guides and course outlines on file for its 
various programs. There is an ongoing effort in curriculum development. All vocational 
grants have provisions for a one-half day substitute teacher for each \/TE teacher to allow 
them to attend inservice and/or to work on curriculum development. 

Level III programs are offered on a semester-long or year-long basis. Some programs are 
available on a double block-time schedule, particularly in the Trade and Industry areas and 
Business. No block time scheduling is available in the Home Economics program. 

ADE competency lists and teacher-designated competencies are written for all vocational 
students. Teacher-designed testing is used as the primary measurement of competency. 
Individual competency attainment records are maintained on the SIMS system. Progress is 
also monitored in a follow-up of program completers one year after graduation. 
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INSTRUCTIONAL DELIVERY 



The majority of competency-attainment activities are completed in the classroom setting. 
Some students participate in on-the-job cooperative programs. This is not true of the Home 
Economics program in which ail instruction is classroom based. The classrooms have 
limited industry standard equipment. 

The ADE strands are used and incorporated into the instruction, particularly career planning 
and cooperative learning activities stressing leadership and decision making. All of the 
essential skills materials have been developed. The district is presently writing business 
management into finance and computer technology, which will be matched to ASAP 
assessment and tracked back into the SIMS system. Emphasis in the district programs is 
on competencies, not courses. 

IVEP Students 

In the Level III programs, including Home Economics, students may demonstrate out of 
skills contained in the "Success Skills" course which is a prerequisite. IVEP students are 
provided alternative competency requirements when necessary. Student progress and 
attainment is tracked through the SIMS program. 

FACILITIES AND EQUIPMENT 

The Home Economics classrooms are fairly traditional. There are two computers, cutting 
tabletops, six cooking stations, a micrcwave and electric stove, standard refrigerator and 
washer/dryer. It is a multi-purpose facility in that the room provides for many activities to 
be occurring simultaneously. It does not reflect industry standards. 

TEACHER QUALIFICATIONS 

All teachers in the Level III programs are vocationally certified. Vocational teachers in the 
district are provided support to attend the Vocational Summer Conference, state 
workshops and, in the case of Home Economics, the STRIVE activities. One teacher is 
currently involved in completing a Master's program. All vocational teachers have technical 
assistance available from an external consultant. 

COMMENTS 

The district considers its Home Economics program to be a "Model" Level III program, but 
it does not meet ADE standards in terms of being industry-current within its facility. The 
curriculum materials being used do incorporate ADE competencies and strands. There is no 
advisory committee for the program, other than the district over-all committee. The district 
meets most of the criteria for a Level III program and could easily be upgraded to 
completely meet that criteria. The Vocational Director has indicated that there is a 
continuous problem with funding and the ability to keep up with change. On the other 
hand, there is a strong commitment to do better and a high teacher interest in maintaining 
strong programs. The director stated that "ADE should decide on program costs and 
allocate dollars based on pre-determined costs." The director further noted that "ADE lacks 
employment data and cannot give good direction to the program(s)". 
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APACHE JUNCTION HS, Apache Junction Unified: Home Economics— Food Preparation 



Program Arct and Characteristics 



LEVEL 
II II! 



I. ORGANIZATION 



1 



Instruction provides: 

• an exploration/skill development for all occupations. 

• skill development ror a cluster of related occupations. 

• skill development for specific occupation(s). 

2. Students served: 

• 100% in the targeted grade level (7-9). 

• In the targeted grade levels (9-10) have previously completed a Level I 
experience. 

• Students have the opportunity to participate in more than one cluster, if 
desired. 

• !n the targeted grade levels (11-12) have completed Level I and II 
experiences. 

3. The experience is a minimum of: 

• one year in length or equivalent to 225 min/week. 

• 360 hours in length. 

4. The experience provides a coherent sequence of instruction: 

• required for all vocational technological programs. 

• each cluster leads to at least one specific Level III experience. 

• leads to employment and advanced training. 
The experience has the support of the local school/district administration. 



5. 
6. 



An active advisory council assists in designing and implementing this 
experience. 



II. CURRICULUM 

7. Curriculum is competency-based with each student's attainment of each 
competency monitored. 

8. Curriculum is designed to deliver all academic and occupational 
competencies (approved by ADE) in: 

• the draft Level I Framework. 

• at least one cluster. 

• specific occupation(s). 



yes 



no 



yes 



yes 



yes 
yes 



no 



no no 



yes 



no 



yes 



no 

yes 

yes yes 



ns yes yes 



no yes yes 



no 



no 



yes 
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Program Area and Characteristics 




LEVEL 




■ 1 




I 


II 


III 


1 


II. CURRICULUM - continued 








1 1 


9. Curriculum includes all six of the Model "Strands." 


no 


no 


no 


1 


10. Curriculum provides for student leadership development, which can include 
the use of vocational student organizations. 


no 


yes 


yes 




III. INSTRUCTIONAL DELIVERY 










1 1. The instruction provides students with experiences in: 








| 1 


• a variety of occupational areas. 


no 








• occupational areas associated with the particular cluster(s). 




no 




1 1 


• specific occupation(s). 






yes 




12. The teacher serves as a facilitator of instruction, managing a variety of 
learning experiences for students. 


yes 


yes 


yes 


1 


13. Students work in groups or teams frequently. 


yes 


yfcs 


yes 


m 1 


14. Instructional activities include industry-realistic situations and materials, and 
are evaluated according to industry-related standards. 


no 


no 


no 




15. The instructional delivery includes occupational experiences which can 
include shadowing and cooperative education. 


no 


ns 


yes 


| 


IV. FACILITIES AND EQUIPMENT 










16. The instructional facility provides a multi- purpose environment with 
appropriate technology and support systems. 


no 


no 


no 




17. The environment simulates one that would be found in business and 
industry. 


no 


no 


no 


| 


18. The equipment and materials utilized in the experience are determined on 
the basis of the curriculum and industry standards. 


no 


no 


no 


| 


V. TEACHER QUALIFICATIONS 










19. The teacher(s) are vocationally certified in: 








1 


• vocational areas or if a team of teachers instruct the Level I experience, at 
least one will be vocationally certified. 


yes 








• appropriate vocational areas related to the specific cluster content. 




yes 




| 


• the specific vocational area. 






yes 




20. The teacher(s) have recent inservice to update and upgrade their teaching 
techniques and content knowledge. 


yes 


yes 


yes 
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Site: Holbrook High School, Holbrook Unified School District 
Program: Business— Administrative Support 



SETTING 

Holbrook High School is located in a unified school district in the north east part of Arizona 
in Navajo County. It was selected as representative of a rural, reservation school offering a 
self-described "non-comprehensive" vocational program that has never received ADE 
"Model site" funding. The district has had a Vocational Director for the past two years. 
This person has resigned effective with the end of the school year to move to a 
metropolitan area. Two teachers work with the business program. 

METHOD 

Staff interviews, including the Vocational Director and one of the two teachers, were 
conducted and a tour of the facility was completed. Other vocational teachers were 
interviewed in a group setting. Conversations were also held with several vocational 
students. Documentation shown but not given to the interviewer, included the following: 

1 . Class/course schedule 

2. Course outline 

3. District Performance Standards Review document 
ORGANIZATION 

Levels Nil 

Level I programs are not available in this district. Occupational clusters are addressed 
through a variety of vocational programs in Level II. These include Business Management 
Technology, Human Services and Industrial Technology. Programs offered included 
Accounting/Computing Occupations, Administrative Support, Carpentry, Building Trades, 
Graphics and Welding. These programs are also offered at Level III (grades 10-12) to 
provide specific vocational training. Level II programs (grades 9-1 1) were also offered in 
Auto Mechanics, General Marketing, Child Care, Clothing/Apparel, Agriculture Business 
Management and Electricity. Level II and Level IN programs are year-long programs. Level II 
courses are considered preparatory. Completion of a Level 1 1 course is not always a 
prerequisite for registration in a Level III program. 

Local and District Support 

There was generally strong overall community support for vocational programs. Board and 
administration support levels were rated at 8-9 on a scale of 10 (with 10 being high). 
Vocational teachers indicated that they felt students would be interested in more 
vocational classes h»!t they were unable to provide them because of funding and/or space 
restrictions. Teachers were not satisfied that the employment needs in the community 
were being met by the present programs. 
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Students also expressed the desire for additional programs, particularly in the areas of 
aviation, careers, computer technology, entrepreneurship, medical and on the job training. 
Students were generally satisfied with their programs and teachers, but not satisfied wit 
books, tools and equipment. 

Advisory Council 

The Vocational Director has established an overall Advisory Committee. The committee 
has worked with the Standards Review and new program presentations. Several members 
of the committee have been guest speakers in vocational classes. 

Holbrook High School also participates on a Joint Curriculum Planning Committee with the 
community college district and has developed written articulation agreements with the 
district. 

CURRICULUM 

The curriculum guides and student enrollment handbooks indicate that vocational programs 
are available to all students. Provisions were made to accommodate IVEP students. The 
curriculum in Level II and III programs utilized ADE curriculum guides as available and 
teacher-selected information. While some work had been done in incorporating the strands, 
these efforts were not as far along as might be desired. The Vocational Director indicated 
that she felt more effort would be directed toward incorporating the strands into the 
curriculum as the articulation agreements were developed. 

VICA, STRIVE and FBLA are available to students to participate in leadership and 
competitive activities. 

ADE competencies are in use in the Level II and III programs, most notably in the 
Administrative Support Program. A project-based approach is used and students are 
encouraged to work at their own pace as much as possible. Some opportunity exists for 
students to develop self-initiated projects. 

Students are monitored and evaluated by teachers. The district completes a follow-up one 
y6ar after graduation. Informal discussions also provide some follow-up information, 
particularly of students who are still in Holbrook or nearby communities. 

INSTRUCTIONAL DELIVERY 

Hands-on, project-based activities are provided to students. In the shop areas (i.e. 
Automotives), community-based projects were in process. In the Administrative Support 
Program, instructional emphasis was on decision-making and performance standards. 

IVEP Students 

Instruction is designed to accommodate students through the IVEP. Students are not 
singled out, but are encouraged to work to the best of their ability. Tracking and 
performance testing is designed to accommodate their individual handicaps. 
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FACILITIES AND EQUIPMENT 



The facilities at Holbrook High School are, for the most part, older facilities in need of 
updating. A Macintosh lab is used as a general computer laboratory and may be utilized in 
many vocational programs. It is not designated for vocational programs and was not 
funded by vocational dollars. It is one of the better classrooms in terms of industry 
standard equipment. The Administrative Support and Accounting areas are fairly well 
equipped and lighting in these areas has been upgraded. The remaining Trade and Industry 
labs (with the exception of Building Trades) are located in older facilities in need of 
housekeeping and general upgrading. Floor space is small and adjacent classroom areas are 
not easily accessible. 

TEACHER QUALIFICATIONS 

All teachers involved with the vocational programs are certified and have appropriate 
endorsements for vocational specialties. 

Teachers are provided assistance in attending the Vocational Conference and curriculum 
dissemination workshops and in making visits to other schools in the area. District 
inservice, through the Vocational Director, has included competency development, testing 
and grading practices and integrated curriculum concerns. 

COMMENTS 

Holbrook High School is in a period of transition. Several long-time teachers have retired 
and/or will retire next year. The Vocational Director has resigned and will be replaced. 
Program development in the Level I area has not yet been addressed. Several more 
traditional vocational programs are being offered and slow progress has been made in 
incorporating the strands into the curriculum. ADE curriculum is being used in conjunction 
with teacher-selected activities. An advisory committee is functioning for the Automotive 
program, and the Administrative Support program appears to have good rapport with the 
community. Articulation efforts are underway, particularly in the Trade and Industry area. 
Additionally, "crosswalks' 1 between business classes are being developed. Teachers are 
not satisfied with the occupational outcomes of the programs and recognize that there is a 
need to update both the program and the equipment. These changes are coming about 
slowly, largely because of funding and facility restrictions. 
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HOLBROOK HIGH SCHOOL, Holbrook Unified: Business Education— Administrative Support 



Program Area and Characteristics 



LEVEL 
I II III 



no 



no 



An active advisory council ...sists in designing and implementing this 
experience. 



I!. CURRICULUM 

7. Curriculum is competency-based with each student's attainment of each 
competency monitored. 

8. Curriculum is designed to deliver all academic and occupational 
competencies {approved by ADE) in: 

• the draft Level I Framework. 

• at least one cluster. 

• specific occupation(s). 



yes 



no 



ns 



no yes 



yes 



yes 



I. ORGANIZATION 

1. Instruction provides: 

• an exploration/skill development for all occupations. 

• skill development for a cluster of related occupations. 

• skill development for specific occupation(s). 

2. Students served: 

• 100% in the targeted grade level (7-9). 

• In the targeted grade levels (S-10) have previously completed a Level I 
experience. 

• Students have the opportunity to participate in more than one cluster, if 
desired. 

• In the targeted grade levels (11-12) have completed Level I and II 
experiences. 

3. The experience is a minimum of: 

• one year in length or equivalent to 225 min/week. 

• 360 hours in length. 

4. The experience provides a coherent sequence of instruction: 

• required for all vocational technological programs. 

• each cluster leads to at least one specific Level III experience. 

• leads to employment and advanced training. 

5. The experience has the support of the local school/district administration. 



yes 



yes 



no 

yes 

yes 

no yes yes 
no yes yes 

no yes yes 

no 



yes 
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Program Area and Characteristics 



LEVEL 
I II III 



tl. CURRICULUM - continued 

9. Curriculum includes all six of the Model "Strands." no no no 

10. Curnculum provides for student leadership development, which can include 

the use of vocational student organizations. no yes yes 

III. INSTRUCTIONAL DELIVERY 

1 1. The instruction provides students with experiences in: 

• a variety of occupational areas. no 

• occupational areas associated with the particular cluster(s). yes 

• specific occupation(s). yes 

12. The teacher serves as a facilitator of instruction, managing a variety of 

learning experiences for students. no yes yes 

13. Students work in groups or teams frequently. no yes yes 

14. Instructional activities include industry-realistic situations and materials, and 

are evaluated according to industry-related standards. no ns yes 

15. The instructional delivery includes occupational experiences which can 

include shadowing and cooperative education. no ns ns 

IV. FACILITIES AND EQUIPMENT 

18. The instructional facility provides a multi-purpose environment with 

appropriate technology and support systems. no no yes 

17. The environment simulates one that would be found in business and 

industry. no yes yes 

18. The equipment and materials utilized in the experience are determined on 

the basis of the curriculum and industry standards. no yes yes 

V. TEACHER QUALIFICATIONS 

19. The teacher(s) are vocationally certified in: 

• vocational areas or if a team of teachers instruct the Level I experience, at 

least one will be vocationally certified. no 

• appropriate vocational areas related to the specific cluster content. yes 

• the specific vocational area. yes 

20. The teacher(s) have recent inservice to update and upgrade their teaching 

techniques and content knowledge. no yes yes 
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APPENDIX C 



Site Visit Comparisons on Program Characteristics 



This appendix contains data used in preparing the analyses of site visits by schools, Model 
level, and program type (comprehensive versus non-comprehensive; Model-funded and not 
Model-funded). 

■ Tables C-1 through C-3 show site visit results on unique program characteristics by level 
for all 1 2 schools. 

■ Tables C-4 through C-6 show site visit results on shared program characteristics by level 
for al! 12 schools. 

* Tables C-7 through C-9 show site visit comparisons between comprehensive and non- 
comprehensive sites for Levels l-lll on unique program characteristics. 

■ Tables C-1 0 through C-1 2 show site visit comparisons between comprehensive and non- 
comprehensive sites for Levels l-lll on shared program characteristics. 

■ Tables C-1 3 through C-1 5 show site visit comparisons between Model-funded and non- 
Model-funded sites for Levels l-lll on unique program characteristics. 

* Tables C-1 6 through C-1 8 show site visit comparisons between Model-funded and non- 
Model-funded sites for Levels l-lll on shared program characteristics. 
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Table C-7. 

Comparison of Comprehensive and Non-comprehensive Sites on "Unique" Characteristics for Level I 



Level I Programs* 





Comprehensive i 
(n = 6) I 


Non-comprehensive 
In = 6) 


Unique Program Characteristics 


yes 


no 


not i 
sure i 


yes 


no 


not 
sure 


Organization (1) Instruction provides an exploration/skill 
development for all occupations. 


3 


3 


— | 


2 


4 


- 


Organization {2) Students served: 100% in the targeted 
grade level (7-9). 


2 


4 




1 


5 




Organization (3) The experience is a minimum of one year in 
length or equivalent to 225 min/week. 


3 


3 


- j 


1 


5 


- 


Organization (4) The experience provides a coherent 
sequence of instruction required for all vocational 
technological programs. 


1 


5 


- 


1 


5 


- 


curriculum ioj curriculum is Qcbiyneu 10 uciivoi an atauciiui/ 
and occupational competencies {approved by ADE) in the 
draft Level 1 Framework. 


3 


1 


2 


1 


5 




Instructional Delivery {11) The instruction provides students 
with experience in a variety of occupational areas. 


5 


1 




2 


4 




Teacher Qualifications {19) The teacher{s) are vocationally 
certified in vocational areas or if a team of teachers instruct 
the Level I experience, at least one will be vocationally 
certified. 


3 


1 


2 


3 

I 


3 






20 


18 


4 


11 

1 


31 





Yes, No, and Not Sure ratings reflect the judgments of site evaluators; Ratings for each criterion has been 
summed across sites 
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Table C-8. 

Comparison of Comprehensive and Non-comprehensive Sites on "Unique" Characteristics for Level II 



Level II Programs* 





Comprehensive 
(n = 6) 


Non-comprehensive 
(n = 6) 


Unique Program Characteristics 


yes 


no 


not i 
sure i 


yes 


no 


not 
sure 


Organization (1) Instruction provides skill development for a 
cluster of related occupations. 


5 


1 




3 


3 




Organization (2-a) Students served: In the targeted grade 
levels (9-10) have previously completed a Level I 
experience. 


2 


4 




1 


5 




Organization (2-b) Students served: Students have the 
opportunity to participate in more than one cluster, if 
desired. 


4 


1 


1 i 


5 




1 


Organization (3) The experience is a minimum of one year in 
length or equivalent to 225 min/week. 


6 






4 


1 


1 


Organization (4) The experience provides a coherent 
sequence of instruction: each cluster leads to at least one 
specific Level III experience. 


3 


3 




4 


2 




Curriculum (8) Curriculum is designed to deliver all academic 
and occupational competencies {approved by ADE) in at 
least one cluster. 


5 


1 




4 


2 




Instructional Delivery {1 1) The instruction provides students 
with experiences in occupational areas associated with the 
particular cluster(s). 


5 


1 




4 


2 




Teacher Qualifications (19) The teacher(s) are certified in 
appropriate vocational areas related to the specific cluster 
content. 


5 


1 




6 








35 


12 


1 i 
i 


31 


15 


2 



Yes, No, and Not Sure ratings reflect the judgments of site evaluators; Ratings for each criterion has been 
summed across sites 
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Table C-9. 



Comparison of Comprehensive and Non-comprehensive Sites on "Unique" Characteristics for Level HI 



Level IN Programs* 





Comprehensive 
(n - 6) i 


Non-comprehensive 
(n - 6) 


1 mini I A DrA/iF^fn r*H^i»o r*tAr! p tir*r 

unique program onaraciensucs 


yes 


no not | 
sure 


yes 


no 


not 
sure 


Organization (1) Instruction provides skill development for 
specific occupation(s). 


6 




6 






Organization (2) Students served: In the targeted grade 
levels (1 1-12) have completed Level I and II experiences. 


1 


5 


2 


4 




Organization (3) The experience is a minimum of 360 hours 
in length. 


6 




6 






Organization (4) The experience provides a coherent 
sequence of instruction: leads to employment and advanced 
training. 


5 


1 


6 






Curriculum (8) Curriculum is designed to deliver all academic 
and occupational competencies (approved by ADE) in 
specific occupationis). 


5 


1 


I 6 






Instructional Delivery (11) The instruction provides students 
with experience in specific occupation(s). 


6 




6 






Teacher Qualifications (19) The teacher(s) are vocationally 
certified in the specific vocational area. 


5 


1 


! 6 








34 


7 1 


i 38 


4 





Yes, No, and Not Sure ratings reflect the judgments of site evaluators; Ratings for each criterion has been 
summed across sites 



15 
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Table C-10. 

Comparison of Comprehensive and Non-Comprehensive Sites on "Shared" Characteristics for Level I 



Level I Programs* 





Comprehensive 
(n = 6) 


Non-comprehensive 
(n = 6) 


Shared Program Characteristics 


yes 


no 


not 
sure ! 


yes 


no 


not 
sure 


Organization (5) The experience h3s the support of the local 
school/district administration. 


3 


- 


3 


3 


3 


- 


Organization (6) An active advisory council assists in 
designing and implementing this experience. 


1 


2 


3 


1 


4 


1 


Curriculum {/) Curriculum is competency-based; 
students'attainment of competencies are monitored. 


3 


- 


3 


1 


4 


1 


Curriculum (9) Curriculum includes all six of the Model 
"Strands." 


1 


2 


3 


1 


4 


1 


Curriculum (10) Curriculum provides for student leadership 
development, which can include the use of vocational 
student organizations. 


1 


1 


4 




6 




Instructional Delivery (12) The teacher serves as a facilitator 
of instruction/ managing a variety of learning experiences 
for students. 


4 




2 


3 


3 




Instructional Delivery (13) Students work in groups or teams 
frequently. 


4 




2 


3 


3 




Instructional Delivery (14) Instructional activities include 
industry-realistic situations/materials, and are evaluated 
according to industry-related standards. 


2 


1 


3 


1 


4 


1 


Instructional Delivery (1 5) The instructional delivery includes 
occupational experiences which can include shadowing and 
cooperative education. 




1 


5 


1 


4 


1 


Facilities and Equipment (16) The instructional facility 
provides a multi-purpose environment with appropriate 
technology and support systems. 


2 


2 


2 


2 


4 




Facilities and Equipment (17) The environment 
simulates. ..business and industry. 


1 


1 


4 


1 


4 


1 


Facilities and Equipment (18) The equipment and materials 
utilized in the experience are determined on the basis of the 
curriculum and industry standards. 


2 


1 


3 


! 1 


4 


1 


Teacher Qualifications (20) The teacher(s) have recent 
inservice to update and upgrade their teaching techniques 
and content knowledge. 


4 




2 


! 3 


3 






28 


1 1 


39 


! 21 


50 


7 



Yes, No, and Not Sure ratings reflect the judgments of site evaluators; Ratings for each criterion has been 
summed across sites 
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Table C-11- 

Comparison of Comprehensive and Non-Comprehensive Sites on "Shared" Characteristics for Level II 



Level II Programs* 





Comprehensive i 
(n = 6) | 


Non-comprehensive 
(n • 6) 


Shared Program Characteristics 


yes 


no 


not i 
sure j 


yes 


no 


not 
sure 


Organization (5} The experience has the support of the local 
scnooi/ Qisinci aarninisirauon. 


5 




1 j 


5 




1 


Organization (6) An active advisory council assists in 
designing and implementing this experience. 


5 


1 


— i 


5 


1 


— 


Curriculum (7) Curriculum is competency-based; 
students'attainment of competencies are monitored. 


6 


- 




4 


- 


2 


Curriculum (9) Curriculum includes ail six of the Model 
"Strands." 


1 


5 




1 


5 




Curriculum (10) Curriculum provides for student leadership 
development, which can include the use of vocational 
student organizations. 


5 


1 




6 






Instructional Delivery (12) The teacher serves as a facilitator 
of instruction, managing a variety of learning experiences 
for students. 


6 


— 


— 


5 


~~ 


1 


Instructional Delivery (13} Students work in groups or teams 
frequently. 


5 




1 


4 




2 


Instructional Delivery (14) Instructional activities include 
industry-realistic situations/materials, and are evaluated 
according to industry-related standards. 


2 


2 


2 


1 


3 


2 


Instructional Delivery (15) The instructional delivery includes 
occupational experiences which can include shadowing and 
cooperative education. 


- 


1 


5 


! 2 


1 


3 


Facilities and Equipment (16) The instructional facility 
provides a multi-purpose environment with appropriate 
technology and support systems. 


5 


1 




! 3 


3 




Facilities and Equipment (17) The environment 
simulates.. .business and industry. 


2 


3 


1 


i 4 


2 




Facilities and Equipment (18) The equipment and materials 
utilized in the experience are determined on the basis of the 
curriculum and industry standards. 


5 


1 




i 4 


2 




Teacher Qualifications (20) The teacher(s) have recent 
inservice to update and upgrade their teaching techniques 
and content knowledge. 


6 






i 6 








53 


15 


10 


! 50 


17 


11 



Yes, No, and Not Sure ratings reflect the judgments of site evaiuators; Ratings for each criterion has been 
summed across sites 
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Table C-12. 

Comparison of Comprehensive and Non-Comprehensive Sites on "Shared" Characteristics for Level HI 



Level III Programs* 





Comprehensive i 
(n = 6) 


Non-comprehensive 
(n = 6) 


Shared Program Characteristics 


yes 


no 


not i 
sure | 


yes 


no 


not 
sure 


Organization (5) The experience has the support of the local 
school/district administration. 


6 






C 






Organization (6) An active advisory council assists in 
designing and implementing this experience. 


5 


1 


— 


5 


1 


— 


Curriculum (7) Curriculum is competency-based; 
students'attainment of competencies are monitored. 


6 


- 


- i 


5 


- 


1 


Curriculum (9) Curriculum includes all six of the Model 
"Strands." 


1 


5 




1 


5 




Curriculum (10) Curriculum provides for student leadership 
development, which can include the use of vocational 
student organizations. 


5 


1 




6 






Instructional Delivery (12) The teacher serves as a facilitator 
of instruction, managing a variety of learning experiences 
for students. 


6 


— 


— 


6 


— 


— 


Instructional Delivery (13) Students work in groups or teams 
frequently. 


6 






6 






Instructional Delivery (14) Instructional activities include 
industry-realistic situations/materials, and are evaluated 
according to industry-related standards. 


4 


2 




4 


2 




Instructional Delivery (15) The instructional delivery includes 
occupational experiences which can include shadowing and 
cooperative education. 


3 


1 


2 


5 




1 


Facilities and Equipment (16) The instructional facility 
provides a multi-purpose environment with appropriate 
technology and support systems. 


5 


1 




4 


2 




Facilities and Equipment (17) The environment 
simulates. ..business and industry. 


3 


3 




4 


2 




Facilities and Equipment (18) The equipment and materials 
utilized in the experience are determined on the basis of the 
curriculum and industry standards. 


5 


1 




4 


2 




Teacher Qualifications (20) The teacher(s) have recent 
inservice to update and upgrade their teaching techniques 
and content knowledge. 


6 






i 6 








61 


15 


2 


! 62 


14 


2 



Yes, No, and Not Sure ratings reflect the judgments of site evaluators; Ratings for each criterion has been 
summed across sites 
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Table C-13. 

Comparison of ADE Model-funded and Nonfunded Sites on "Unique" Characteristics for Level I 



Level I Programs* 







ADE-funded 
(n = 6) 






Nonfunded 
(n = 6) 


Unique Program Characteristics 


yes 


r.o 


not 
sure 


yes 


no not 
sure 


Organization (1) Instruction provides an exploration/skill 
development for all occupations. 


4 


2 


- 


1 


5 


Organization (2) Students served: 1 00% in the targeted 
grade level (7-9). 


- 


6 


- 


3 


3 


Organization (3) The experience is a minimum of one year in 
length or equivalent to 225 min/week. 


4 


2 






6 — 


Organization (4) The experience provides a coherent 
sequence of instruction required for all vocational 
technological programs. 


1 


5 


- 


1 


5 - 


f*t i rrir*i tli tm f*i trrini tli tm rip^innpr! tn rfpliv/pr all *aparlpmip 
1 1 1 vcUiui 1 1 \o/ wUi I iuuiui 1 1 io ucoiyi icu iu uciivci ait doauciiiii* 

and occupational competencies (approved by ADE) in the 
draft Level 1 Framework. 




o 


i 
i 


i 
i 


A 1 


Instructional Delivery (11) The instruction provides students 
with experience in a variety of occupational areas. 


5 


1 




2 


4 


Teacher Qualifications (19) The teacher(s) a.e vocationally 
certified in vocational areas or if a team of teachers instruct 
the Level I experience, at least one will be vocationally 
certified. 


4 


1 


1 


! 2 


3 1 




21 


19 


1 


! 10 


30 1 



Yes, No, and Not Sure ratings reflect the judgments of site evaluators; Ratings for each criterion has been 
summed across sites 



19 
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Table C-14. 

Comparison of AOE Model-funded and Nonfunded Sites on "Unique" Characteristics for Level I! 



Level l\ Programs* 







ADE-funded I 
(n = 6) 




Nonfunded 
(n = 6) 




Unique Program Characteristics 


yes 


no not i 
sure i 


yes 


no 


not 
sure 


Organization (1) Instruction provides skill development for a 
cluster of related occupations. 


5 


1 - ] 


3 


3 


- 


Organization (2-a) Students served: In the targeted grade 
levels (9-10) have previously completed a Level I 
experience. 




6 — i 


3 


3 




Organization (2-b) Students served: Students have the 
opportunity to participate in more than one cluster, if 
desired. 


6 


- - 


3 


1 


2 


Organization (3) The experience is a minimum of one year in 
length or equivalent to 225 min/week. 


5 


- 1 


5 


1 




Organization (4) The experience provides a coherent 
sequence of instruction: each cluster leads to at least one 
specific Level HI experience. 


3 


3 


4 


2 




Curriculum (8) Curriculum is designed to deliver all academic 
and occupational competencies (approved by ADE) in at 
least one cluster. 


5 


1 


4 


2 




Instructional Delivery (11) The instruction provides students 
with experiences in occupational areas associated with the 
particular cluster(s). 


6 




3 


3 




Teacher Qualifications (19) The teacher(s) are certified in 
appropriate vocational areas related to the specific c'uster 
content. 


6 




5 


1 






35 


12 1 


30 


16 


2 



Yes, No, and Not Sure ratings reflect the judgments of site evaluators; Ratings for each criterion has been 
summed across sites 
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Table C-15. 

Comparison of ADE Model-funded and Nonfunded Sites on "Unique" Characteristics for Level III 



Level III Programs* 







ADE-funded 
{n = 6) 






Nonfunded 
(n = 6} 


Unique Program Characteristics 


yes 


no 


not 
sure 


yes 


no not 
sure 


Organization {1} Instruction provides skill development for 
specific occupation(s). 


6 


- 


- 


6 


- - 


Organization (2) Students served: In the targeted grade 
levels (11-12) have completed Level I and II experiences. 


2 


4 




1 


5 - 


Organization (3) The experience is a minimum of 360 hours 
in length. 


6 




- 


6 


- - 


Organization (4) The experience provides a coherent 
sequence of instruction: leads to employment and advanced 
training. 


6 






5 


1 


Curriculum (8) Curriculum is designed to deliver all academic 
and occupational competencies {approved by ADE) in 
specific occupation(s). 


6 






5 


1 - 


Instructional Delivery (11) The instruction provides students 
with experience in specific occupation(s). 


6 






! 6 




Teacher Qualifications (19) The teacher(s) are vocationally 
certified in the specific vocational area. 


6 






! 5 


1 




38 


4 




! 34 


7 1 



Yes, No, and Not Sure ratings reflect the judgments of site evaluators; Ratings for each criterion has been 
summed across sites 



ERIC 
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Table C-16. 



Comparison of ADE Model-funded and Nonfunded Sites on "Shared" Characteristics for 

Level I 



Level I Programs* 







ADE-funded 
(n = 6) 






Nonfunded 
(n = 6) 




Shared Program Characteristics 


yes 


no 


not i 
sure i 


yes 


no 


not 
sure 


Organization (5) The experience has the support of the local 

Qphnnl/Hictript ^HmintQtrJitinn 

dU lUUw UloUlOl aUIIIM Uo 11 Q UUI 1. 


2 


2 


2 


4 


-j 


1 


Organization (6) An active advisory council assists in 
designing and implementing this experience. 


2 


2 


2 




3 


3 


Curriculum (7) Curriculum is competency-based; 
students'attainmerit of competencies are monitored. 


3 


2 


1 i 


1 


2 


3 


Curriculum (9) Curriculum includes all six of the Model 
"Strands." 


2 


3 


1 


- 


3 


3 


Curriculum (10) Curriculum provides for student leadership 
development, which can include the use of vocational 
student organizations. 


1 


3 


2 


- 


4 


2 


Instructional Delivery (12) The teacher serves as a facilitator 
of instruction, managing a variety of learning experiences 
for students. 


3 


2 


1 


3 


3 


- 


Instructional Delivery (13) Students work in groups or teams 
frequently. 


3 


2 


1 


4 


1 


1 


Instructional Delivery (14) Instructional activities include 
industry-realistic situations/materials, and are evaluated 
according to industry-related standards. 


2 


2 


1 


1 


3 


2 


Instructional Delivery (15) The instructional delivery includes 
occupational experiences which can include shadowing and 
cooperative education. 


1 


2 


3 


I - 


3 


3 


Facilities and Equipment (16) The instructional facility 
provides a multi-purpose environment with appropriate 
technology and support systems. 


2 


2 


2 


! 2 


4 


— 


Facilities and Equipment (17) The environment 
simulates. ..business and industry. 


1 


2 


3 


! 1 


3 


2 


Facilities and Equipment (18) The equipment and materials 
utilized in the experience are determined on the basis of the 
curriculum and industry standards. 


2 


2 


2 


j 1 


3 


2 


Teacher Qualifications (20) The teacher(s) have recent 
inservice to update and upgrade their teaching techniques 
and content knowledge. 


2 


2 


2 


i 5 


1 






26 


28 


23 


! 22 


34 


22 



Yes, No, and Not Sure ratings reflect the judgmerts of site evaluators; Ratings for each criterion has been 
summed across sites 



9 

ERIC 



22 
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Table C-17. 



Comparison of ADE Model-funded and Nonfunded Sites on "Shared* 1 Characteristics for 

Level II 



Level II Programs* 







ADE-funded 
(n = 6) 






Nonfunded 
(n = 6) 




Shared Program Characteristics 


yes 


no 


not i 
sure ] 


yes 


no 


not 
sure 


Organization (5) The experience has the support of the local 
school/district administration. 


• 5 




1 i 


5 




1 


Organization (6) An active advisory council assists in 
designing and implementing this experience. 


5 


1 




5 


1 




Curriculum (7) Curriculum is competency-based; 
students'attainment of competencies are monitored. 


4 


- 


2 


6 


- 


- 


Curriculum (9) Curriculum includes all six of the Model 
"Strands." 


2 


4 


- 


- 


6 


- 


Curriculum (10) Curriculum provides for student leadership 
development, which can include the use of vocational 
student organizations. 


6 


- 


- 


5 


1 


- 


Instructional Delivery (12) The teacher serves as a facilitator 
of instruction, managing a variety of learning experiences 
for students. 


5 


- 


1 


6 


- 


- 


Instructional Delivery (13) Students work in groups or teams 
frequently. 


4 




2 


5 




1 


Instructional Delivery (14) Instructional activities include 
industry-realistic situations/materials, and are evaluated 
according to industry-related standards. 


3 


2 


1 


j - 


3 


3 


Instructional Delivery (15) The instructional delivery includes 
occupational experiences which can inc'ude shadowing and 
cooperative education. 


2 


1 


3 


j - 


1 


5 


Facilities and Equipment (16) The instructional facility 
provides a multi-purpose environment with appropriate 
technology and support systems. 


6 




— 


i 2 


4 


- 


Facilities and Equipment (17) The environment 
simulates. ..business and industry. 


4 


1 


1 


j 2 


4 




Facilities and Equipment (18) The equipment and materials 
utilized in the experience are determined on the basis of the 
curriculum and industry standards. 


5 


1 




j 4 


2 




Teacher Qualifications (20) The teacher(s) have recent 
inservice to update and upgrade their teaching techniques 
and content knowledge. 


6 






! 6 








57 


10 


11 


! 46 


22 


10 



Yes, No, and Not Sure ratings reflect the judgments of site evaluators; Ratings for each criterion has been 
summed across sites 



ERIC 
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Table C-18. 

Comparison of ADE Model-funded and Nonfunded Sites on "Shared" Characteristics for 
Level II 

Level III Programs* 







ADE-funded 
(n = 6) 






Nonfunded 
(n = 6) 




Shared Program Characteristics 


ves 


no 


not 
sure 


yes 


no 


not 
sure 


Organization (5) The experience has the support of the local 
school/district administration. 


6 






5 






Organization (6) An active advisory council assists in 
designing and implementing this experience. 


5 


1 


— 


5 


1 


— 


Curriculum (7) Curriculum is competency-based; 
students'attainment of competencies are monitored. 


5 


- 


1 j 


6 


- 


- 


Curriculum (9) Curriculum includes all six of the Mode! 
" Strands." 


2 


4 


- 


- 


6 


- 


Curriculum (10) Curriculum provides for student leadership 
development, which can include the use of vocational 
student organizations. 


6 


- 


- 


5 


1 




Instructional Delivery (12) The teacher serves as a facilitator 
of instruction, managing a variety of learning experiences 
for students. 


6 


- 


- 


6 


- 


- 


Instructional Delivery (13) Students work in groups or teams 
frequently. 


6 






6 






Instructional Delivery (14) Instructional activities include 
industry-realistic situations/materials, and are evaluated 
according to industry-related standards. 


6 


- 


- 


2 


4 




Instructional Delivery (15) The instructional delivery includes 
occupational experiences which can include shadowing and 
cooperative education. 


5 


- 


1 


3 


1 


2 


Facilities and Equipment (16) The instruccional facility 
provides a multi-purpose environment with appropriate 
technology and support systems. 


6 




— 


3 


3 


— 


Facilities and Equipment (17) The environment 
simulates. ..business and industry. 


5 


1 




2 


4 




Facilities and Equipment (18) The equipment and materials 
utilized in the experience are determined on the basis of the 
curriculum and industry standards. 


5 


1 




4 


2 




Teacher Qualifications (20) The teacher(s) have recent 
inservice to update and upgrade their teaching techniques 
and content knowledge. 


6 






6 








69 


7 


2 


! 54 


22 


2 



Yes. No, and Not Sure ratings reflect the judgments of site evaluators; Ratings for each criterion has been 
summed across sites 



9 

ERIC 
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APPENDIX D 



LEA Performance Standards Procedures Survey Results 



All results reported in this appendix reflect the total number of surveys analyzed (N = 
128). Results by occupational area are available as separate documents. As a general rule, 
respondents were asked to check only one answer For a few questions, respondents were 
asked to "check all that apply." For these questions, numbers and percentages exceed 1 28 
and 100 percent, respectively. 

Results are reported by section in the order in which they appeared on the survey. 
Questions have been paraphrased slightly to assist readability. 

SECTION I: PERFORMANCE STANDARDS -MEASURING ACADEMIC SKILLS (BASIC) 

Question 1.1: Survey recipients were asked to indicate whether they used a formal 
procedure to identify academic skills in al! courses in their program. 

100 (78 percent) indicated they identified academic skills in aN courses using the same 



14 (1 1 percent) said they identified academic skills in al] courses but used different 
procedures. 

13(10 percent) said academic skills are not formally identified for every course. 

1 respondent (>1 percent) did not respond. 

Question 1.2: Survey recipients were asked to indicate what procedure is us id to identify 
academic skills related to their particular occupational program. Of the 1 28 respondents: 

6 (5 percent) indicated the academic curriculum was analyzed. 

12 (9 percent) said the VTE program curriculum was analyzed. 

17 (13 percent) said a test was analyzed. 

41 (32 percent) indicated that teacher judgement was used. 

26 (20 percent) said Arizona's Essential Skills were used. 

6 (5 percent) specified other procedures were used, such as the Mesa Public Schools 



12 (9 percent) gave multiple responses, such as a test was analyzed in addition to 
using teacher judgement or Arizona's Essential Skills. 

8 (6 percent) did not respond. 



procedure. 



Test. 
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Question 1.3: Pertaining to identification of basic academic skills, survey respondents were 
asked in what areas were specific academic skills identified. 

110 (86 percent) said communication such as reading and writing. 

1 1 1 {87 percent) indicated mathematics. 
67 (52 percent) indicated science. 

5 (4 percent) indicated other areas such as language. 

1 1 (9 percent) did not respond. 

Question 1.4: Survey respondents were asked to indicate who was primarily involved in 
the process to identify the specific academic skills associated with the courses in their 
particular program. 

73 (57 percent) indicated the individual VTE course program instructor(s). 

15 (12 percent) said the school/district VTE personnel only. 

I (>1 percent) indicated school/district personnel from academic areas only. 

12 (9 percent) indicated interdisciplinary school/district teams. 

6 (5 percent) said ADE-provided resources. 

6 (5 percent) Indicated other processes were used such as Mesa Public Schools Test. 

5 (4 percent) gave multiple responses such as individual VTE course program 
instructor and ADE-provided resources. 

10 (8 percent) did not respond. 

Question 1.5: Pertaining to student assessment methods for basic academic skills, are the 
same methods used in ever/ course to assess students' academic gains? 

I I 5 (90 percent) answered yes. 
1 2 (9 percent) answered no. 

1 (>1 percent) did not respond. 
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Question 1.6: Pertaining to student assessment methods for basic academic skills, does 
every course use a test to assess students' academic gains? 

1 20 (94 percent) answered yes. 

7 (5 percent) answered no. 

1 (>1 percent) did not respond. 

Question 1.7: Respondents were asked to indicate, pertaining to student assessment 
methods for basic academic skills, what kinds of test(s) is/are used to measure student 
gain. 

16 (13 percent) said ASAP (Arizona Student Assessment Program). 
10 (8 percent) said ASU Academic Skills Assessment for Model Sites. 
21 (16 percent) indicated a district test (e.g., a CRT). 

10 (8 percent) said norm referenced test such as ITBS/TAP. 
75 (59 percent) said teacher developed test. 

17 (13 percent) indicated other tests such as Kingman High School Test, Mesa Public 
Schools Test and ADE provided tests. 

6 (5 percent) did not respond. 

Question 1.8: Pertaining to student assessment methods for basic academic skills, are all 
students in every course pre-post tested? 

109 (85 percent) said yes. 

14 (1 1 percent) said no. 

5 (4 percent) did not respond. 
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SECTION I: PERFORMANCE STANDARDS — MEASURING ACADEMIC SKILLS (ADVANCED) 

Question 1.1: Survey recipients were asked to indicate whether they used a formal 
procedure to identify academic skills in all courses in their program. 

96 (75 percent) indicated they identified academic skills in all courses using the same 
procedure. 

1 1 (9 percent) said they identified academic skills in all courses but used different 
procedures. ♦ 

15(12 percent) said academic skills are not formally identified for every course. 

6 recipients (5 percent) did not respond. 

Question 1.2: Survey recipients were asked to indicate what procedure is used to identify 
academic skills related to their particular occupational program. 

7 (5 percent) said the academic curriculum was analyzed. 

12 (9 percent) indicated the VTE program curriculum was analyzed. 
16(13 percent) said a test was analyzed. 

43 (34 percent) said that teacher judgement was used. 

22 (17 percent) indicated Arizona's Essential Skills were used. 

6 (5 percent) said other procedures such as the Mesa Public Schools Test and ADE 
provided materials were used. 

8 (6 percent) gave multiple responses such as academic curriculum, VTE curriculum 
and a test was analyzed. 

14 recipients (1 1 percent) did not respond. 

Question 1.3: Survey respondents were asked to indicate what areas specific academic 
skills were identified. 

107 (84 percent) indicated communication (reading and writing), 

108 (84 percent) indicated mathematics. 
65 (51 percent) said science. 

5 (4 percent) identified other areas such as language. 
14(11 percent) did not respond. 
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Question 1.4: Survey respondents were asked to indicate who was primarily involved in 
the process to identify the specific academic skills associated with the courses in their 
particular program. 

70 (55 percent) indicated the individual VTE course program instructor(s). 

15(12 percent) said the school/district VTE personnel only, 

1 (> 1 percent) indicated school/district personnel from academic areas only, 

1 2 (9 percent) indicated interdisciplinary school/district teams. 

6 (b percent) said ADE-provided resources. 

6 (5 percent) said other processes were used such as the Mesa Public Schools Test. 

4 (3 percent) gave multiple responses, the most common one being individual VTE 
course/program instructor in addition to an "other" method. 

14 recipients (1 1 percent) did not respond. 

Question 1.5: Respondents were asked, pertaining to student assessment methods for 
advanced academic skills, if the same methods are used in every course to assess 
students' academic gains. 

1 12 (88 percent) answered yes. 

12 (9 percent) answered no. 

4 recipients (3 percent) did not answer. 

Question 1.6: Survey recipients were asked to indicate, pertaining to student assessment 
methods for advanced academic skills, if every course uses a test to assess students' 
academic gains. 

116 (91 percent) indicated yes. 

8 (6 percent) indicated no. 

4 (3 percent) of the recipients did not respond. 
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Question 1.7: Respondents were asked to indicate, pertaining to student assessment 
methods for advanced academic skills, what kind of test(s) is/are used to measure student 
gain. 

14 (1 1 percent) indicated ASAP (Arizona Student Assessment Program). 

10 (8 percent) said ASU Academic Skills Assessment for Model Sites. 

19 (15 percent) indicated a district test (e.g., CRT). 

9 (7 percent) use a norm referenced test such as ITBS/TAP. 

72 (56 percent) use a teacher-devsloped test. 

17 (13 percent) indicated some other type of test such as the Mesa Public Schools 
Test, a test developed by Kingman High School, ADE provided tests, and the Tech 
Prep Consortium. 

8 (6 percent) did not respond. 

Question 1.8: Respondents were asked, pertaining to student assessment methods for 
advanced academic skills, are all students in every course pre-post tested? 

105 (82 percent) answered yes. 

14 (1 1 percent) answered no. 

9 (7 percent) did not respond. 

SECTION II: PERFORMANCE STANDARDS-MEASURING OCCUPATIONAL COMPETENCY 

Question 2.1: Survey respondents were asked to indicate how occupational 
competencies/tasks were identified for their particular program. 

111 (87 percent) said competencies/tasks were identified for aH courses using the 
same procedure. 

14(11 percent) said competencies/tasks were identified for all courses using different 
procedures. 

1 (> 1 percent) said competencies are not formally identified for every course. 

2 (2 percent) did not respond. 
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Question 2.2: Survey recipients were asked to indicate the procedure that is used to 
identify occupational competencies/tasks for the courses in their particular program. 

24 (19 percent) indicated the VTE program curriculum was analyzed. 
1 (> 1 percent) responded that a test was analyzed. 

22 (17 percent) said that teacher judgement was used. 

65 (51 percent) indicated that the ADE validated competency/task lists were used. 

3 (2 percent) said that other occupational competencies/tasks were used such as 
competencies and input from local advisory council. 

8 (6 percent) gave multiple responses such as a combination of the VTE program 
curriculum, teacher judgement and the ADE validated competency/task list. 

5 (4 percent) did not respond. 

Question 2.3: Respondents were asked to indicate who was primarily involved in the 
process to identify the specific occupational competencies and tasks associated with the 
courses in their particular program. 

58 (45 percent) said the VTE course instructor(s) in the program. 

7 (5 percent) indicated the school/district VTE teams. 

15(12 percent) indicated the local advisory council/Program advisory committee. 

25 (20 percent) said the ADE competency /task lists were used. 

4 (3 percent) indicated others who were primarily involved such as the FAA or the 
local community college (EAC) business instructors and the business teachers in 
Graham/Greenlee counties. 

12 (9 percent) gave a multiple response, for example, the school district VTE teams 
and the local advisory council/Program advisory committee. 



7 (5 percent) did not respond. 



Question 2.4: Pertaining to student assessment methods, survey respondents were asked 
if similar methods are used in every course to assess student attainment of occupational 
competencies and tasks. 



1 12 (88 percent) said yes. 

1 2 (9 percent) said no. 

4 (3 percent) did not respond. 




Question 2.5: In relation to student assessment methods, survey recipients were asked if 
every course uses a course-based performance measure of occupational competencies, for 
example a project or skill demonstration. 

1 1 3 (88 percent) indicated yes. 

9 (7 percent) indicated no. 

6 (5 percent) did not respond. 

Question 2.6: Pertaining to student assessment methods, survey respondents were asked 
to indicate what other methods, if any, are used to determine the attainment of 
occupational competencies and tasks. 

52 (41 percent) said they used on-the-job supervisor evaluations. 

47 (37 percent) indicated they used student evaluations. 

113 (88 percent) used ^acher-developed paper and pencil tests. 

125 (98 percent) said they used teacher observation and judgement of student 
performance. 

1 1 (9 percent) indicated they used other methods, such as ADE developed "pencil and 
paper" tests. 

1 (>1 percent) did not respond. 

Question 2.7: In relation to the requirements of their particular program, survey 
respondents were asked how many individual courses constitute this program in their 
school. 

82 (64 percent) said 1-3 courses. 

39 (30 percent) indicated 4-6 courses. 

7 (5 percent) answered 7 or more courses. 

Question 2.8: Pertaining to the requirements of their particular program, survey recipients 
were asked the typical length of a course. 

8 (6 percent) said one semester. 

1 13 (88 percent) indicated two semesters. 

5 (4 percent) gave a multiple response indicating one and two semesters. 

2 (2 percent) did not respond. 
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Question 2.9: Survey respondents were asked to define course completer in their particular 
program. 

45 (35 percent) said all students enrolled in the first grading period who finish the 
course. 

69 (54 percent) said only students who receive credit for the course. 

1 1 (9 percent) gave other definitions such as a student who completes 80 percent or 
more of the competencies. 

2 (2 percent) gave a multiple answer consisting of the first two responses. 

1 (>1 percent) did not respond. 

Question 2.10: Survey respondents were asked, pertaining to the requirements of their 
program, how many courses at a minimum, must a program completer have successfully 
passed. 

104 (81 percent) indicated 1-3 courses. 
18 (14 percent) said 4-6 courses. 

2 (2 percent) said 7 or more courses. 

1 (> 1 percent) gave a multiple response. 

3 (2 percent) did not respond. 

Question 2.11: In reference to the requirements of their program, survey respondents were 
asked if program completers are required to obtain applied and occupational experience in 
the industry for which they are preparing as part of the program. 

58 (45 percent) said yes. 

67 (52 percent) said no. 

3 (2 percent) did not respond. 



Question 2.12: In relation to their particular program requirements, survey recipients were 
asked what types of applied experiences are required. 

6 (5 percent) indicated apprenticeship program(s). 

30 (23 percent) said clinical experience, paid or unpaid. 

22 (17 percent) indicated cooperative education. 

49 (38 percent) said school-based work simulation(s). 

6 (5 percent) indicated other requirements such as a Supervised Occupational 
Experience (SOE) program (an ownership or placement type). 

57 (45 percent) did not respond. 

SECTION 111: PERFORMANCE STANDARDS -MEASURING CONTINUED ENROLLMENT 

Question 3.1: Survey recipients were asked to identify the definition of "course 
enrollment" for their particular program. 

34 (27 percent) defined course enrollment as all students who register for a course 
within the first grading period. 

60 (47 percent) defined course enrollment as all students enrolled in the course 
throughout the term. 

30 (23 percent) defined course enrollment as all students who successfully complete 
course requirements. 

3 (2 percent) described course enrollment as "most of the students who register 
within the first grading period and stay the whole year are course completers." 

1 (> 1 percent) did not respond. 

Question 3.2: Survey respondents were asked whether their school has a management 
information system (MIS) that allows them to track students that leave the course or 
program. 

63 (49 percent) said yes. 

63 (49 percent) said no. 

2 (2 percent) did not respond. 
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Question 3.3: Survey respondents were asked, in their opinion, how much emphasis is put 
on providing teachers with in-service activities or training that addresses reasons why 
students drop out of school. 

5 (4 percent) said it is a high priority with frequent activities. 

62 (48 percent) said it is a moderate priority with some activities. 

61 (48 percent) said it is a low priority with few or no activities. 

Question 3.4: Respondents were asked, in their opinion, is the emphasis from Question 
3.3 appropriate? 

64 (50 percent) said yes. 

63 (49 percent) said no. 

1 (>1 percent) did not respond. 

Question 3.5: Survey recipients were asked how effective they considered their program 
to be in keeping kids enrolled in school. 

64 (50 percent) said very effective. 

60 (47 percent) said somewhat effective. 

4 (3 percent) said not very effective. 

SECTION IV: PERFORMANCE STANDARDS — IVEP STUDENTS 

Question 4.1: Survey recipients were asked to what extent do they believe special 
populations such as IVEP students are represented in their program. 

31 (24 percent) indicated it was overrepresented or too many. 

92 (72 percent) believed it was proportionately represented. 

5 (4 percent) said it was underrepresented or too few. 
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Question 4.2: Survey respondents were asked to what extent IVEPs are used by 
instructors in their particular program. 

48 (38 percent) said often used. 

66 (52 percent) indicated sometimes used. 

12 (9 percent) said rarely used. 

2 (2 percent) did not respond. 

Question 4.3: Respondents were asked to indicate to what extent are instructors in their 
program involved in developing IVEPS. 

82 (64 percent) said often involved. 

27 (21 percent) indicated sometimes involved. 

16 (13 percent) said rarely involved. 

3 (2 percent) did not respond. 

Question 4.4: Survey respondents were asked to what extent instructors in their program 
coordinate instruction with other instructors working with IVEP students. 

26 (20 percent) indicated frequent coordination. 

73 (57 percent) said some coordination. 

25 (20 percent) indicated infrequent coordination. 

4 (3 percent) did not respond. 

Question 4.5: According to the Performance Standards, IVEP students are expected to 
perform as well as regular students. Survey respondents were asked, in their opinion, how 
many instructors in their program believe that this is a realistic expectation. 

22 (17 percent) said all instructors. 

44 (34 percent) indicated most instructors. 

56 (44 percent) said less than half of the instructors. 

6 (5 percent) did not respond. 



ERIC 



D ' 1 2 1 n r 



Question 4.6: Respondents were asked if basic academic, skills for IVEP students are 
measured in the same way as for regular students. 

1 1 1 (87 percent) said yes. 

1 1 (9 percent) said no. 

2 (2 percent) gave a multiple response. 

4 (3 percent) did not respond. 

Question 4.7: Survey recipients were asked if more advanced academic skills for IVEP 
students are measured in the same way as for regular students. 

1 13 (88 percent) said yes. 

9 (7 percent) said no. 

2 (2 percent) gave a multiple response. 

4 (3 percent) did not respond. 

Question 4.8: Respondents were asked if the attainment of occupational competency for 
IVEP students is measured in the same way as for regular students. 

109 (85 percent) said yes. 
15(12 percent) said no. 

4 (3 percent) gave no response. 

SECTION V: PREPARING AND CONDUCTING THE LOCAL EVALUATION 

Question 5.1: Survey recipients were asked if their district's VTE director acts as the local 
evaluation coordinator. 

1 10 (86 percent) said yes. 
16 (13 percent) said no. 

1 (> 1 percent) indicated they did not know or the question was not applicable. 
1 (> 1 percent) did not respond. 



Question 5,2: Regarding assessment and data collection, respondents were asked to 
indicate what types of training and support were provided for the teachers of their 
program. 

93 (73 percent) said district in-service training. 

74 (58 percent) indicated district support such as administrative and clerical 
assistance. 

39 (30 percent) indicated state-sponsored training. 
35 (27 percent) said ADE staff assistance. 

7 (5 percent) did not respond. 

Question 5.3: Survey respondents were asked how often evaluation information was 
shared with the teachers for their particular program. 

71 (55 percent) indicated regularly. 

40 (31 percent) said occasionally. 

10 (8 percent) said seldom. 

5 (4 percent) indicated never. 

2 (2 percent) did not respond. 

Question 5.4: Survey respondents were asked if there was a local evaluation team for their 
program. 

1 1 2 (88 percent) indicated yes. 
14 (1 1 percent) said no. 

1 (>1 percent) gave a multiple response. 

1 (>1 percent) did not respond. 

Question 5.5: Survey respondents were asked how many members were on the local 
evaluation team for their program. 

Of the 112 respondents who indicated they did have a team, the range was from 2 to 
20 and the average was 6. 
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Question 5. 6. a: Respondents were asked if the team induded a teacher representative. 

1 14 (89 percent) said yes. 

0 (0 percent) said no. 

14 (1 1 percent) did not respond. 

Question 5.6. b: Survey recipients were asked if the team included a business and industry 
representative. 

108 (84 percent) indicated yes. 

2 (2 percent) said no. 

18 (14 percent) did not respond. 

Question 5.6.c: Survey respondents were asked if the team included a special population 
representative. 

106 (83 percent) said yes. 

6 (5 percent) indicated no. 

16 (13 percent) did not respond. 

Question 5.7: Survey recipients were asked if the team received training for its role. 
90 (70 percent) indicated yes. 

19 (15 percent) said no. 

4 (3 percent) said they did not know or the question was not applicable. 

15(12 percent) did not respond. 

Question 5.8: Respondents were asked if the performance standards were not met, did the 
local evaluation team play a key role in the development of a local improvement plan. 

88 (69 percent) said yes. 

17 (13 percent) said no. 

3 (2 percent) said they did not know or the question was not applicable. 

20 (16 percent) did not respond. 
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Question 5.9: Survey respondents were asked if it was clearly understood that team 
members who disagreed with the district report could file a minority report. 

103 (80 percent) said yes. 

7 (5 percent) said no. 

3 (2 percent) indicated they dio not know or the question was not applicable. 
15 (12 percent) did not respond. 
Question 5.10: Survey recipients were asked if any team member(s) filed a minority report. 
9 (7 percent) indicated yes. 
99 (77 percent) said no. 

6 (5 percent) did not know or said the question was not applicable. 
14 (1 1 percent) did not respond. 

SECTION VI: USER EVALUATION OF PERFORMANCE STANDARDS SYSTEM 

Question 6.1: Survey recipients were asked if they agree that the Performance Standards 
requiring that students demonstrate gains in basic and advanced academic skills are 
important for evaluating VTE programs in their school/district. 

22 (17 percent) strongly agree. 
74 (58 percent) agree. 

23 (18 percent) disagreed. 

9 (7 percent) strongly disagreed. 

Question 6.2: Survey recipients were asked if they agreed that their school /district can 
accurately determine whether each student in a VTE course makes achievement gains in 
basic and advanced academic skills during a school year. 

15(12 percent) strongly agreed. 

87.5 (68 percent) agreed. 

20.5 (16 percent) disagreed. 

5 (4 percent) strongly disagreed. 
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Question 6.3: Survey recipients were asked if they agreed that the Performance Standards 
requiring that students attain occupational competency are important for evaluating VTE 
programs in their school/district. 

37.5 (29 percent) indicated they strongly agreed. 

75.5 (59 percent) said they agreed. 

9 (7 percent) said they disagreed. 

5 (4 percent) strongly disagreed. 

1 (<1 percent) did not respond. 

Question 6.4: Survey recipients were asked if they agreed that their school/district can 
accurately determine whether course completers attain 80% of the occupational 
competencies for each course. 

29 (23 percent) strongly agreed. 

87 (68 percent) agreed. 

9 (7 percent) disagreed. 

3 (2 percent) strongly disagreed. 

Question 6.5: Survey recipients were asked if they agreed that their school/district can 
accurately determine whether program completers attain 80% of the occupational 
competencies for each program. 

24 (19 percent) strongly agreed. 

89 (70 percent) agreed. 

12 (9 percent) disagreed. 

3 (2 percent) strongly disagreed. 

Question 6.6: Survey rec vients were asked if they agreed that the Performance Standard 
requiring that students stay enrolled in or graduate from school is important for evaluating 
VTE programs in their school/district. 

19 (15 percent) strongly agreed. 

57 (45 percent) agreed. 

37 (29 percent) disagreed. 

15 (12 percent) strongly disagreed. 
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Question 6.7: Survey recipients were asked if they agreed that their school/district can 
accurately determine whether 90% of the students in each VTE course stay enrolled in or 
complete high school. 

42 (33 percent) strongly agreed. 

72.5 (57 percent) agreed. 

9.5 (7 percent) disagreed. 

4 (3 percent) strongly disagreed. 

Question 6.8: Survey recipients were asked if they agreed that tracking regular and IVEP 
students separately is important for evaluating VTE programs in their school/district. 

9 (7 percent) strongly agreed. 

55.5 (43 percent) agreed. 

56.5 (44 percent) disagreed. 

7 (5 percent) strongly disagreed. 

Question 6.9: Survey recipients were asked if they agreed that their school/district can 
accurately identify the number of both regular and IVEP students in each VTE program. 

44.5 (35 percent) strongly agreed. 

74.5 (58 percent) agreed. 

6 (5 percent) disagreed. 

2 (2 percent) strongly disagreed. 

1 (> 1 perr^nt) did not respond. 

Question 6.10: Survey respondents were asked if they agreed that the local evaluation 
teams play an important role in their school/district's VTE program. 

26 (20 percent) strongly agreed. 

72.5 (57 percent) agreed. 

24.5 (19 percent) disagreed. 

4 (3 percent) strongly disagreed. 

1 {< 1 percent) did not respond. 
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Question 6,11: Survey respondents were asked if they agreed that their district would 
have a local evaluation team for each VTE program even if there were no state 
involvement in the programs. 

13 (10 percent) strongly agreed. 

61.5 (48 percent) agreed. 

47.5 (37 percent) disagreed. 

4 (3 percent) strongly disagreed. 

2 (2 percent) did not respond. 

Question 6.12: Survey respondents were asked if they agreed that it is reasonable to 
expect school districts to have a local evaluation team with a teacher representative, a 
business and industry representative, and special populations representative for each VTE 
program. 

25 (20 percent) strongly agreed. 

74 (58 percent) agreed. 

17 (13 percent) disagreed. 

1 2 (9 percent) strongly disagreed. 

Question 6,13: Survey respondents were asked if they agreed that it is reasonable to 
expect school districts to maintain and submit the Performance Standards reports required 
by federal legislation and ADE. 

6 (5 percent) strongly agreed. 

80 (63 percent) agreed. 

30 (23 percent) disagreed. 

1 1 (9 percent) strongly disagreed. 

1 (> 1 percent) did not respond. 
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Question 6.14: Survey respondents were asked if they agreed that once the "bugs" are 
worked out, the Arizona VTE standards will be useful for making important improvements 
in their school/district VTE programs, 

15 (12 percent) strongly agreed. 

78 (61 percent) agreed. 

21 (16 percent) disagreed. 

14(11 percent) strongly disagreed. 

Question 6-15: Survey respondents were asked if they agreed that the costs, such as time 
and money, of keeping the state-required student performance records and reporting them 
to ADE will be justified by the benefits to their school/district. 

5 (4 percent) strongly agreed. 

48 (38 percent) agreed. 

44 (34 percent) disagreed. 

29 (23 percent) strongly disagreed. 

2 (2 percent) did not respond. 

Question 6.16: Survey respondents were asked if they agreed that they would recommend 
to school districts in other states that they use a VTE performance standards system like 
Arizona's. 

3 (2 percent) strongly agreed. 
57 (45 percent) agreed. 

39 (30 percent) disagreed. 

25 (20 percent) strongly disagreed. 

4 (3 percent) did not respond. 
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ADDITIONAL COMMENTS 



Finally, Section VII of the survey requested additional comments regarding any aspect of 
the Performance Standards system. For the most part, comments hove been reproduced 
verbatim except for minor editing for spelling, grammar, and clarity. 

■ Instructional time lost is more valuable than the performance standards. 

« Vocational teachers need release time on a weekly or monthly basis to keep up on the 
record keeping. 

■ As teachers We need three things: 1 . More time 2. More time 3. More time 

■ Teachers need time to do record keeping. It would not be unreasonable for each 
vocational teacher to have one period/day to do record keeping. 

■ I realize that accountability is important but I believe that all accountability should be 
local. The time and effort spent on documenting state standards takes time away from 
time spent with students. I believe my program is exceptional, I've been recognized for 
my teaching, I pour my life into my program. Why do I have to spend so much time 
recording/documenting /verifying the achievement of 130+ students. Please - free us 
to do our best! 



■ [My] school district tries to follow all State Department guidelines; however, I feel 
each district could come up with good guidelines to evaluate their own programs. In 
order for us to receive materials (such as curriculum) to be used in evaluations, we are 
required to attend workshops which I'm sure are the State Department's answer to 
getting their federal funding. All of these conferences at large convention centers in 
nice motels cost money that could be used on students in the classroom. 

■ The ADE Performance standards reporting process is cumbersome and very time 
consuming, both on the part of the course/program instructor and the local evaluation 
coordinator. It must be made less cumbersome. 

■ 1. To cut to the chase: We don't presently have an evaluation team. 

2. Secondly for the mere $2,500 I expect to get for doing all these state required 
performance standards this sure seems like a lot of paperwork. 

3. As tight as funds are now, who is going to buy the file cabinets in which to keep 
all the students quizzes, exams, lab exams, and demonstration sheets to effectively 
evaluate each student? 
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4. It would be particularly helpful to make sure the evaluation folks knew what 
agricultural education courses taught. Most folks figure its something to do with cows. 

5. If teachers could use some of the "A" monies derived initially from their programs 
to use in their departments they would look more favorably on all this. 
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The current Performance Standards measurement are a very poor way to evaluate the 
competency attainment of individual students. This form can be made to say what the 
instructor wishes it to say! I think this is a very poor evaluation tool and it should be 
restructured. My local evaluation team members also feel that this system is 
worthless. 

Performance standards and measures are absolutely essential to assure the quality of 
vocational educational delivery. Uniform standards seem to be necessary for all 
programs to assure equality of vocational education delivery across the board. 
However, in health occupations, and specifically in nursing assistant programs, there is 
a federally mandated curriculum which must be followed to allow students to take the 
State certification exam to become certified nursing assistants at the end of their 
program. Because of this special requirement in the area of nursing assistant, there is 
a very limited amount of modification that can be made to the curriculum for special 
needs students. Nursing Assistant programs all include a clinical component in which 
students go to the hospital and work "hands-on" with patients under the license of 
their instructor. The instructor takes responsibility for the actions of her students (and 
virtually all HOE instructors are female!!) and her nursing license could be in jeopardy. 
There is a very strong motivation, on part of instructors, to be sure that students can 
perform all the skills in which they have been trained. This is the reason that there are 
few special needs students in the health occupations programs unless the programs 
are large enough to have multiple instructors who can support the special needs of 
these students in health care occupations. It is certainly reasonable to expect districts 
to submit a Performance Standards report to ADE. The nature of the report appears to 
be beyond reasonable expectations for the following reasons: 

1. Definitions are unclear. A "program completer" means different things to different 
people, even at the ADE. This opens the door to confusion and frustration on 
everyone's part. "Basic Academic Skills" and "More Advanced Academic Skills" mean 
different things to everyone. 

2. It seems unfair to hold districts responsible for contacting every student for a 
follow-up study. There needs to be a standard whereby every reasonable follow-up 
effort is made and statistics are calculated based on those students who can be found. 
To withhold funding because a district serves a transient population seems very 
arbitrary. There are certainly mathematical tools which are available to allow the use of 
a representative sampling of program completers (wh .ever they are) and still have 
valid statistics. 

3. The amount of time spent on program standards is like having a North Central 
evaluation every year. Voc Ed teachers are expected to spend many volunteer hours 
each year at this task. Many of those teachers who have been around for awhile feel 
that "this too shall pass" (like the program standards of the late 80s) and are 
frustrated with the necessity of spending time in such seemingly endless and valueless 
paperwork instead of improving curriculum and/or pedagogical skills. From what I have 
heard, many of the teachers who are also certified to teach in an academic area are 
considering a move to the area of their academic certification when an opening occurs. 
Many of these are very strong teachers and it bodes ill for voc ed to be facing the 
possible loss of these talented individuals. 
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■ The Performance Standards and Measures instrument we are currently using is one of 
the worst instruments ! have had to work with. Here are the reasons why: 

1. It does not accurately measure what it is supposed to because no one knows for 
sure what the criteria might be (i.e., A teacher may at his/her own discretion, just cut 
down the amount of competencies to just a few, say 3-5, and have every single 
student in the program meet standards— they are not standardized throughout a 
school, or a district, or in the state. 

2. A student can meet every so called "definition" except one and never be counted in 
the ^atistics as a "success" (i.e., A student may transfer in after the 40-day count 
and successfully complete or transfer to another class after completing 1st semester 
only and never be counted as meeting standards. 

3. A student is told in every vocational class in which they are enrolled to declare that 
program area as their choice and end up being counted in the wrong area because they 
don't know which one to choose— the one they are taking a one-semester class in, or 
the one they are taking three classes consecutively in????? so, they chose both!! 

4. So much of the information is criss-nrossed so that they meet the criteria for one 
but not the other, etc., etc., etc. and therefore can't be counted at all. 

5. Who's definition of "demonstrated gain" are we using? I assume that a gain of one 
point can be counted as gain. 

6. On the basis of this questionnaire, I assume that we can have our very own 
definition of "course" and "program" completer. And I'm sure that if this is true, how 
much does it vary from program to program just in one school (sic). How can you 
compare? 

7. We were told to "not include" anyone who falls through the cracks if they were not 
meeting any one of the specified criteria which brings our count down very low. What 
impact does this have on the accuracy of the statistics in our standards??? 

8. Just exactly what is an acceptable "advanced related academic skill"? For an IVEP 
student, it might be simply one of your more complex basic skills. Do you have to 
redefine it for every student??? 

9. Part of the statistics are gathered on the 40th day count, if you get 25 new 
students signing up for your one year course at the semester, you can't count them. 
Not a good idea! 

10. This evaluation form tries to accommodate for every little exception by excluding 
them, but ends up being so vague that it really accounts for a minute portion of what 
a program is really like. 

1 1 . The Program Standards evaluation form does not account for a student who takes 
your class 1st semester only, successfully completes it, leaves school and goes into 
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that fieid of work— obviously one of your successes, but it never shows in your 
program standards report! 

12. Because of the "constant changing" of this information (students switching from 
job to job) f the whole picture may change prior to your submitting it and be totally 
inaccurate, (followup statistics) 

13. By teacher observation, it seems that some students should definitely be listed as 
"disadvantaged" but are not. Some in my opinion should not be but definitely are. 

14. The members of our local advisory committee feel that this evaluative instrument 
is one of the worst and feels it is worthless and does not reflect the overall picture of 
any of our programs. 

15. The Program Standards instrument needs major revision— the key is to K.I.S.S.— 
Keep It Simple S 

■ We are changing our academic skills assessment this year. We will be using SIMS, a 
district developed assessment system tied to the State Essential Skills. 

We discontinued using the AIMS system. We now track competencies and produce 
IVEP'S using the Vocational Data Management System. The system is being used for 
reporting Performance Standards. We are piloting a scanning competency process... 
this spring. We would not have achieved our first year accomplishments or been able 
to work on our improvement plan without using rural technical assistance staff. 
The ADE needs not only to tell you what to do, but develop a system for doing it. This 
is expensive and time consuming, 

■ Since this is a small school district with a total 9-1 2 enrollment under 200 students, 
the instructors are the people who must do all of the tracking and testing and 
valuating of programs— This takes a great deal of time away from course-related 
instruction. We find that keeping track of students and their progress is easy because 
we see them all the time. Completing the ADE Performance Standards is done so the 
district can maintain approved programs and be eligible for state and federal funds. 
The work is difficult and time-consuming. As Voc Ed. coordinator, I find the time spent 
on the standards to be extremely taxing, as I also teach 6 classes a day. I wish the 
processes, especially the academic standards, could be streamlined and tied in with 
district ASAP testing. 

■ I hope it's possible to combine much of the tracking/testing for academic skill 
attainment to each district's ASAP testing. Also, students reelly get tired of taking 
tests they perceive as meaningless— between ASAP, ITBS/TAP and PONS. So it's 
difficult to get an accurate measure of a student's abilities. 

■ We should only be evaluating targeted programs. 

■ Many students do mission work, sit out a year, or have other personal requirements to 
complete after graduation and so they drop "our" percentage and make our program 
appear unsuccessful. 
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I am new to this district and I still do not know all the details involved in the inner 
workings of the Vocational Program within all school districts. At this time I am the 
Health Occupation Department, so the answers on this questionnaire are from one 
person's point of view. 
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Morrison Institute for Public Policy 



Established in 198 1 through a gift from the Morrison family of Gilbert, Arizona, Morrison Institute 
for Public Policy is an Arizona State University (ASU) resource for public policy research, expertise, and 
insight. The Institute conducts research on public policy matters, informs policy makers and the public 
about issues of importance to Arizona, and advises leaders on choices and actions. A center in the School 
of Public Affairs (College of Public Programs), Morrison Institute helps make ASU's resources accessible 
by bridging the gap between the worlds of scholarship and public policy. 

The Institute's primary functions are to offer a variety of services to public and private sector clients 
and to pursue its own research agenda. Morrison Institute's services include policy research and analysis, 
program evaluation, strategic planning, public policy forums, and support of citizen participation in public 
affairs. The Institute also serves ASU's administration by conducting research pertinent to a variety of 
university affairs. 

Morrison Institute's researchers are some of Arizona's most experienced and well-known policy 
analysts. Their wide-ranging experiences in the public and private sectors and in policy development at the 
local, state, and national levels ensure that Morrison Institute's work is balanced and realistic. The 
Institute's interests and expertise span such areas as education, urban growth, the environment, human 
services, and economic development. 

The Institute's funding comes from grants and contracts from local, state, and federal agencies and 
private sources. State appropriations to Arizona State University and endowment income enable the 
Institute to conduct independent research and to provide some services pro bono. 



Morrison Institute for Public Policy 
School of Public Affairs 
Arizona State University 
Tempe, Arizona 85287-4405 
(602) 965-4525 
(602) 965-9219 (fax) 
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